2024-04-14T14:51:49.784 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-14T14:51:49.785 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi --description orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} --owner scheduled_teuthology@teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060 -- /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/orig.config.yaml 2024-04-14T14:51:49.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T14:51:49.897 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060 branch: main description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: ceph-qa@ceph.com first_in_suite: false job_id: '7646060' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: a971fd5f5719038c70b817bafacd6d5400b1d888 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: a971fd5f5719038c70b817bafacd6d5400b1d888 extra_packages: - cephadm selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 05c9d46e7de5db2cb434c177a6bbfc56f472d09b owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 6509 sha1: a971fd5f5719038c70b817bafacd6d5400b1d888 sleep_before_teardown: 0 subset: 21/64 suite: orch suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_05c9d46e7de5db2cb434c177a6bbfc56f472d09b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 05c9d46e7de5db2cb434c177a6bbfc56f472d09b targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgw8XwaCi5zWKb5A5BMOx1xFLxK4BHifsyKLM7h5Co0jY5I4GGsxgFWR4Vj/Yj3l12z506Fc5Z38/URcB+vWw= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPRRi40gafZuRBeII8+chWhAdPK7DY92ZmEkVZmZuS+dKcKDeSShFwO2r8zcYdbgZVmIRbxW1lUF6djnR096H/s= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-08_20:08:14 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-14T14:51:49.897 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_05c9d46e7de5db2cb434c177a6bbfc56f472d09b/qa; will attempt to use it 2024-04-14T14:51:49.898 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_05c9d46e7de5db2cb434c177a6bbfc56f472d09b/qa/tasks 2024-04-14T14:51:49.898 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-14T14:51:49.899 INFO:teuthology.task.internal:Checking packages... 2024-04-14T14:51:49.924 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a971fd5f5719038c70b817bafacd6d5400b1d888' 2024-04-14T14:51:49.925 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-14T14:51:49.925 INFO:teuthology.packaging:ref: None 2024-04-14T14:51:49.925 INFO:teuthology.packaging:tag: None 2024-04-14T14:51:49.925 INFO:teuthology.packaging:branch: main 2024-04-14T14:51:49.925 INFO:teuthology.packaging:sha1: a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T14:51:49.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-04-14T14:51:50.174 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3080.gc2d8c315 2024-04-14T14:51:50.176 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-14T14:51:50.182 INFO:teuthology.task.internal:no buildpackages task found 2024-04-14T14:51:50.182 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-14T14:51:50.217 INFO:teuthology.task.internal:Saving configuration 2024-04-14T14:51:50.228 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-14T14:51:50.267 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T14:51:50.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 14:47:02.453171', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgw8XwaCi5zWKb5A5BMOx1xFLxK4BHifsyKLM7h5Co0jY5I4GGsxgFWR4Vj/Yj3l12z506Fc5Z38/URcB+vWw='} 2024-04-14T14:51:50.323 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 14:47:02.451179', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPRRi40gafZuRBeII8+chWhAdPK7DY92ZmEkVZmZuS+dKcKDeSShFwO2r8zcYdbgZVmIRbxW1lUF6djnR096H/s='} 2024-04-14T14:51:50.323 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-14T14:51:50.331 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-04-14T14:51:50.331 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-04-14T14:51:50.331 INFO:teuthology.run_tasks:Running task console_log... 2024-04-14T14:51:50.425 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd557dddf70>, signals=[15]) 2024-04-14T14:51:50.425 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-14T14:51:50.451 INFO:teuthology.task.internal:Opening connections... 2024-04-14T14:51:50.451 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-14T14:51:50.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:51:50.564 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-14T14:51:50.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:51:50.651 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-14T14:51:50.658 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-14T14:51:50.676 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-14T14:51:50.677 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-14T14:51:50.734 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-04-14T14:51:50.734 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-04-14T14:51:50.734 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-04-14T14:51:50.734 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T14:51:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T14:51:50.736 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-14T14:51:50.759 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-14T14:51:50.776 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-14T14:51:50.776 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-04-14T14:51:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T14:51:50.834 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-04-14T14:51:50.834 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T14:51:50.834 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T14:51:50.834 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T14:51:50.834 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-14T14:51:50.867 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-14T14:51:50.877 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-14T14:51:50.917 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-14T14:51:50.918 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-14T14:51:50.920 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-14T14:51:50.936 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-14T14:51:50.942 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-14T14:51:50.942 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-14T14:51:50.977 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-14T14:51:51.165 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-14T14:51:51.220 INFO:teuthology.run_tasks:Running task kernel... 2024-04-14T14:51:51.235 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-14T14:51:51.236 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-14T14:51:51.236 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-14T14:51:51.236 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-14T14:51:51.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T14:51:51.253 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-14T14:51:51.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T14:51:51.254 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-14T14:51:51.309 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-361.el9.x86_64 2024-04-14T14:51:51.310 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-361.el9.x86_64 2024-04-14T14:51:51.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-04-14T14:51:51.312 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-361.el9.x86_64 2024-04-14T14:51:51.312 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-361.el9.x86_64 2024-04-14T14:51:51.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-04-14T14:51:54.075 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 6.7 MB/s | 8.1 MB 00:01 2024-04-14T14:51:54.126 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 6.6 MB/s | 8.1 MB 00:01 2024-04-14T14:51:55.842 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.5 MB 00:00 2024-04-14T14:51:56.126 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.5 MB 00:00 2024-04-14T14:51:56.774 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-14T14:51:59.941 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.0 MB 00:03 2024-04-14T14:52:01.826 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-14T14:52:07.226 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-14T14:52:07.440 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 19 MB 00:09 2024-04-14T14:52:08.893 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-14T14:52:10.495 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-14T14:52:11.508 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-14T14:52:13.016 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-14T14:52:15.504 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-04-14T14:52:15.870 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-14T14:52:16.107 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:01 ago on Sun 14 Apr 2024 02:52:15 PM UTC. 2024-04-14T14:52:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T14:52:18.229 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.230 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T14:52:18.231 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T14:52:18.232 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.233 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T14:52:18.234 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.235 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.236 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.237 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T14:52:18.651 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-14T14:52:18.652 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-14T14:52:18.755 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T14:52:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:52:18.775 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 77 M 2024-04-14T14:52:18.776 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 123 M 2024-04-14T14:52:18.776 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T14:52:19.312 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 542 kB/s | 214 kB 00:00 2024-04-14T14:52:20.087 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-435.el9.x86_64.rpm 13 MB/s | 15 MB 00:01 2024-04-14T14:52:20.659 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-14T14:52:20.913 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 17 MB/s | 34 MB 00:01 2024-04-14T14:52:21.196 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-435.el9.x86_6 15 MB/s | 28 MB 00:01 2024-04-14T14:52:21.199 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T14:52:21.199 INFO:teuthology.orchestra.run.smithi045.stdout:Total 32 MB/s | 77 MB 00:02 2024-04-14T14:52:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T14:52:22.332 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T14:52:22.332 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T14:52:22.908 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.909 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.910 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T14:52:22.911 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.912 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.913 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.914 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.915 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T14:52:22.916 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T14:52:23.250 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-14T14:52:23.250 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-14T14:52:23.354 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T14:52:23.354 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T14:52:23.354 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T14:52:23.360 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T14:52:23.360 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-435.el9 baseos 214 k 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-435.el9 baseos 15 M 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-435.el9 baseos 34 M 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-435.el9 baseos 28 M 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-04-14T14:52:23.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:52:23.362 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 77 M 2024-04-14T14:52:23.362 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 123 M 2024-04-14T14:52:23.362 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T14:52:23.603 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-435.el9.x86_64.rpm 1.4 MB/s | 214 kB 00:00 2024-04-14T14:52:23.903 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-core-5.14.0-435.el9.x86_64.rpm 34 MB/s | 15 MB 00:00 2024-04-14T14:52:24.311 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-modules-5.14.0-435.el9.x86_64.rpm 39 MB/s | 34 MB 00:00 2024-04-14T14:52:24.620 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-modules-core-5.14.0-435.el9.x86_6 27 MB/s | 28 MB 00:01 2024-04-14T14:52:24.620 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T14:52:24.620 INFO:teuthology.orchestra.run.smithi071.stdout:Total 61 MB/s | 77 MB 00:01 2024-04-14T14:52:25.286 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T14:52:25.655 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T14:52:25.655 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T14:52:25.971 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T14:52:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-14T14:52:26.660 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T14:52:26.660 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T14:52:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:52:29.149 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T14:52:29.630 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:52:29.764 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:52:29.780 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-435.el9.x86_64 1/4 2024-04-14T14:52:29.863 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:52:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:52:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:52:34.839 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:52:34.881 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:52:37.462 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:52:37.579 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:52:39.980 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:52:42.301 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.064 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.064 INFO:teuthology.orchestra.run.smithi045.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-14T14:53:11.064 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:53:11.111 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.429 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-14T14:53:11.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:53:11.429 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:53:11.822 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:53:11.823 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T14:53:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:11.859 INFO:teuthology.orchestra.run.smithi071.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-14T14:53:11.859 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:53:11.905 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:12.148 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T14:53:12.238 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:12.238 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-435.el9.x86_64 1/4 2024-04-14T14:53:12.238 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-435.el9.x86_64 2/4 2024-04-14T14:53:12.238 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-435.el9.x86_64 3/4 2024-04-14T14:53:12.255 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.256 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T14:53:12.256 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-14T14:53:12.256 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-14T14:53:12.256 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-14T14:53:12.256 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-14T14:53:12.256 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-04-14T14:53:12.257 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T14:53:12.340 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.341 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T14:53:12.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-04-14T14:53:12.378 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T14:53:12.379 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-04-14T14:53:12.380 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-14T14:53:12.380 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-435.el9.x86_64 2024-04-14T14:53:12.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-14T14:53:12.678 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-435.el9.x86_64 4/4 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-435.el9.x86_64 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:53:12.679 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T14:53:13.007 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T14:53:13.024 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-04-14T14:53:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:13.108 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T14:53:13.108 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-435.el9.x86_64 2024-04-14T14:53:13.108 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-14T14:53:13.108 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-14T14:53:13.108 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-14T14:53:13.109 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-04-14T14:53:13.109 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-04-14T14:53:13.189 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-435.el9.x86_64 2024-04-14T14:53:13.190 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-435.el9.x86_64 2024-04-14T14:53:13.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 68.el9 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-14T14:53:13.230 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8264254 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-14T14:53:13.231 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-14T14:53:13.232 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-04-14T14:53:13.232 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T14:53:13.232 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-04-14T14:53:13.232 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-14T14:53:13.232 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-435.el9.x86_64 2024-04-14T14:53:13.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-14T14:53:13.843 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-04-14T14:53:14.443 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-14T14:53:14.471 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-04-14T14:53:14.474 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-04-14T14:53:14.530 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.VqvvJpvouf 2024-04-14T14:53:14.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VqvvJpvouf 2024-04-14T14:53:14.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.VqvvJpvouf 2024-04-14T14:53:14.746 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.VqvvJpvouf is 6KB 2024-04-14T14:53:14.794 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.VqvvJpvouf 2024-04-14T14:53:14.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-04-14T14:53:14.876 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-14T14:53:14.876 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-14T14:53:14.876 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-14T14:53:14.876 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-14T14:53:14.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-14T14:53:15.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-04-14T14:53:15.247 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-14T14:53:15.301 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-04-14T14:53:15.303 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-04-14T14:53:15.318 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.ivS9xZfbLK 2024-04-14T14:53:15.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ivS9xZfbLK 2024-04-14T14:53:15.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.ivS9xZfbLK 2024-04-14T14:53:15.524 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.ivS9xZfbLK is 6KB 2024-04-14T14:53:15.573 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.ivS9xZfbLK 2024-04-14T14:53:15.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-04-14T14:53:15.651 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-14T14:53:15.651 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-14T14:53:15.651 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-14T14:53:15.651 INFO:teuthology.orchestra.run.smithi071.stdout:15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64.conf 2024-04-14T14:53:15.652 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-435.el9.x86_64 2024-04-14T14:53:15.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-04-14T14:53:45.054 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-14T14:53:45.055 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:53:45.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:53:45.805 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-14T14:53:45.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-14T14:53:45.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:54:03.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-14T14:54:12.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:54:12.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:54:45.842 DEBUG:teuthology.orchestra.remote:timed out 2024-04-14T14:54:54.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-14T14:54:54.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:54:55.176 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-14T14:54:55.592 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-14T14:54:55.592 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-435.el9.x86_64"... 2024-04-14T14:54:55.592 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-04-14T14:54:55.606 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-435.el9.x86_64 2024-04-14T14:54:55.606 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-14T14:54:55.606 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-14T14:54:55.607 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-14T14:54:56.607 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-14T14:54:56.607 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-14T14:54:56.689 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-04-14T14:54:56.715 DEBUG:teuthology.parallel:result is None 2024-04-14T14:55:12.563 DEBUG:teuthology.orchestra.remote:timed out 2024-04-14T14:55:24.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:55:24.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:55:24.856 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-14T14:55:25.235 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:55:25.235 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-435.el9.x86_64"... 2024-04-14T14:55:25.235 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-14T14:55:25.250 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-435.el9.x86_64 2024-04-14T14:55:25.251 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-435.el9.x86_64 vs 5.14.0-435.el9.x86_64 2024-04-14T14:55:25.251 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-14T14:55:25.251 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-14T14:55:26.252 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-14T14:55:26.252 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-14T14:55:26.322 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-04-14T14:55:26.364 DEBUG:teuthology.parallel:result is None 2024-04-14T14:55:26.365 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-14T14:55:26.372 INFO:teuthology.task.internal:Creating test directory... 2024-04-14T14:55:26.372 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-14T14:55:26.375 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-14T14:55:26.394 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-14T14:55:26.405 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-14T14:55:26.455 INFO:teuthology.task.internal:Creating archive directory... 2024-04-14T14:55:26.456 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-14T14:55:26.458 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-14T14:55:26.499 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-14T14:55:26.505 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-14T14:55:26.505 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-14T14:55:26.520 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-14T14:55:26.556 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T14:55:26.567 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T14:55:26.570 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T14:55:26.581 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-14T14:55:26.582 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-14T14:55:26.589 INFO:teuthology.task.internal:Configuring sudo... 2024-04-14T14:55:26.589 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-14T14:55:26.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-14T14:55:26.652 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-14T14:55:26.661 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-14T14:55:26.661 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-14T14:55:26.683 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-14T14:55:26.708 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T14:55:26.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T14:55:26.826 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T14:55:26.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T14:55:26.951 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T14:55:26.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-14T14:55:27.019 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T14:55:27.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-14T14:55:27.114 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T14:55:27.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-14T14:55:27.241 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T14:55:27.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-14T14:55:27.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-14T14:55:27.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-14T14:55:27.357 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T14:55:27.407 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T14:55:27.889 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-14T14:55:27.897 INFO:teuthology.task.internal:Starting timer... 2024-04-14T14:55:27.897 INFO:teuthology.run_tasks:Running task pcp... 2024-04-14T14:55:27.950 INFO:teuthology.run_tasks:Running task selinux... 2024-04-14T14:55:28.000 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-14T14:55:28.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-04-14T14:55:28.055 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-04-14T14:55:28.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-04-14T14:55:28.115 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-04-14T14:55:28.117 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-14T14:55:28.117 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-04-14T14:55:28.142 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-04-14T14:55:28.143 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-04-14T14:55:28.176 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-04-14T14:55:28.176 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive'} 2024-04-14T14:55:28.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-14T14:55:28.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T14:55:28.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-14T14:55:28.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T14:55:28.244 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-14T14:55:28.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-04-14T14:55:28.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-04-14T14:55:28.319 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-14T14:55:28.329 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-14T14:55:28.344 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-14T14:55:28.346 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi071.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-14T14:59:51.874 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com')] 2024-04-14T14:59:51.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:59:51.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:59:51.971 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-14T14:59:52.061 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-14T14:59:52.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-14T14:59:52.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T14:59:52.147 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-14T14:59:52.224 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-14T14:59:52.225 INFO:teuthology.run_tasks:Running task clock... 2024-04-14T14:59:52.236 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-14T14:59:52.236 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-14T14:59:52.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T14:59:52.239 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-14T14:59:52.239 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T14:59:52.269 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-14T14:59:52.287 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-14T14:59:52.295 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-14T14:59:52.311 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-14T14:59:52.317 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-04-14T14:59:52.331 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-04-14T14:59:52.340 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-04-14T14:59:52.348 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-14T14:59:52.354 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-04-14T14:59:52.364 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-14T14:59:52.370 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-14T14:59:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-14T14:59:52.412 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-14T14:59:52.415 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T14:59:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-14T14:59:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.416 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.437 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-14T14:59:52.441 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T14:59:52.441 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-14T14:59:52.441 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.441 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.442 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.442 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T14:59:52.442 INFO:teuthology.run_tasks:Running task install... 2024-04-14T14:59:52.450 DEBUG:teuthology.task.install:project ceph 2024-04-14T14:59:52.450 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a971fd5f5719038c70b817bafacd6d5400b1d888'}, 'extra_packages': ['cephadm']} 2024-04-14T14:59:52.450 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'a971fd5f5719038c70b817bafacd6d5400b1d888'} 2024-04-14T14:59:52.450 INFO:teuthology.task.install:Using flavor: default 2024-04-14T14:59:52.457 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-14T14:59:52.457 INFO:teuthology.task.install:extra packages: [] 2024-04-14T14:59:52.458 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a971fd5f5719038c70b817bafacd6d5400b1d888', 'tag': None, 'wait_for_package': False} 2024-04-14T14:59:52.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T14:59:52.460 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a971fd5f5719038c70b817bafacd6d5400b1d888', 'tag': None, 'wait_for_package': False} 2024-04-14T14:59:52.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T14:59:52.613 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/ 2024-04-14T14:59:52.613 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2778.ga971fd5f 2024-04-14T14:59:52.718 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/ 2024-04-14T14:59:52.718 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2778.ga971fd5f 2024-04-14T14:59:52.793 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-14T14:59:52.793 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T14:59:52.793 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-14T14:59:52.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-14T14:59:52.821 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a971fd5f5719038c70b817bafacd6d5400b1d888/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-14T14:59:52.827 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/a971fd5f5719038c70b817bafacd6d5400b1d888/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-14T14:59:52.827 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T14:59:52.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-14T14:59:52.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-14T14:59:52.856 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a971fd5f5719038c70b817bafacd6d5400b1d888/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-14T14:59:52.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-14T14:59:52.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-14T14:59:52.969 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-14T14:59:53.004 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-14T14:59:53.035 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-04-14T14:59:53.036 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-04-14T14:59:53.070 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-04-14T14:59:53.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-04-14T14:59:53.280 INFO:teuthology.orchestra.run.smithi071.stdout:74 files removed 2024-04-14T14:59:53.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw 2024-04-14T14:59:53.317 INFO:teuthology.orchestra.run.smithi045.stdout:74 files removed 2024-04-14T14:59:53.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2024-04-14T14:59:54.155 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 152 kB/s | 83 kB 00:00 2024-04-14T14:59:54.190 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 155 kB/s | 83 kB 00:00 2024-04-14T14:59:54.724 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-04-14T14:59:54.839 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-04-14T14:59:55.316 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-14T14:59:55.531 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-14T14:59:56.344 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 8.2 MB/s | 8.1 MB 00:00 2024-04-14T14:59:56.533 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - AppStream 8.4 MB/s | 8.1 MB 00:00 2024-04-14T14:59:58.027 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 8.1 MB/s | 2.5 MB 00:00 2024-04-14T14:59:58.367 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.5 MB 00:00 2024-04-14T15:00:00.307 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-04-14T15:00:00.384 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-14T15:00:02.083 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-14T15:00:05.310 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 19 MB 00:03 2024-04-14T15:00:07.052 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-14T15:00:09.101 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-14T15:00:10.744 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-14T15:00:12.320 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-14T15:00:18.907 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 21 MB 00:06 2024-04-14T15:00:24.113 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-14T15:00:26.791 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.792 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.793 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:00:26.794 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.795 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:00:26.796 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.797 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.798 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.799 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.800 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:00:26.801 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:00:27.300 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 10 M 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.4 M 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.1 M 2024-04-14T15:00:27.310 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 5.4 M 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 21 M 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 25 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 746 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 501 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-14T15:00:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 5.2 M 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 45 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 135 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 161 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 321 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 298 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 99 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-14T15:00:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout:Install 30 Packages 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-04-14T15:00:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:27.314 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 59 M 2024-04-14T15:00:27.314 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:00:28.201 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-14T15:00:29.064 INFO:teuthology.orchestra.run.smithi045.stdout:(1/32): ceph-base-19.0.0-2778.ga971fd5f.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-04-14T15:00:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:(2/32): ceph-common-19.0.0-2778.ga971fd5f.el9.x 13 MB/s | 21 MB 00:01 2024-04-14T15:00:29.289 INFO:teuthology.orchestra.run.smithi045.stdout:(3/32): ceph-selinux-19.0.0-2778.ga971fd5f.el9. 112 kB/s | 25 kB 00:00 2024-04-14T15:00:29.331 INFO:teuthology.orchestra.run.smithi045.stdout:(4/32): libcephfs2-19.0.0-2778.ga971fd5f.el9.x8 11 MB/s | 746 kB 00:00 2024-04-14T15:00:29.431 INFO:teuthology.orchestra.run.smithi045.stdout:(5/32): libradosstriper1-19.0.0-2778.ga971fd5f. 3.4 MB/s | 501 kB 00:00 2024-04-14T15:00:29.515 INFO:teuthology.orchestra.run.smithi045.stdout:(6/32): python3-ceph-argparse-19.0.0-2778.ga971 543 kB/s | 45 kB 00:00 2024-04-14T15:00:29.649 INFO:teuthology.orchestra.run.smithi045.stdout:(7/32): ceph-radosgw-19.0.0-2778.ga971fd5f.el9. 5.1 MB/s | 10 MB 00:01 2024-04-14T15:00:29.674 INFO:teuthology.orchestra.run.smithi045.stdout:(8/32): python3-ceph-common-19.0.0-2778.ga971fd 848 kB/s | 135 kB 00:00 2024-04-14T15:00:29.758 INFO:teuthology.orchestra.run.smithi045.stdout:(9/32): librgw2-19.0.0-2778.ga971fd5f.el9.x86_6 12 MB/s | 5.2 MB 00:00 2024-04-14T15:00:29.783 INFO:teuthology.orchestra.run.smithi045.stdout:(10/32): python3-cephfs-19.0.0-2778.ga971fd5f.e 1.2 MB/s | 161 kB 00:00 2024-04-14T15:00:29.808 INFO:teuthology.orchestra.run.smithi045.stdout:(11/32): python3-rados-19.0.0-2778.ga971fd5f.el 2.3 MB/s | 321 kB 00:00 2024-04-14T15:00:29.834 INFO:teuthology.orchestra.run.smithi045.stdout:(12/32): python3-rbd-19.0.0-2778.ga971fd5f.el9. 3.9 MB/s | 298 kB 00:00 2024-04-14T15:00:29.867 INFO:teuthology.orchestra.run.smithi045.stdout:(13/32): python3-rgw-19.0.0-2778.ga971fd5f.el9. 1.2 MB/s | 99 kB 00:00 2024-04-14T15:00:29.967 INFO:teuthology.orchestra.run.smithi045.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 673 kB/s | 107 kB 00:00 2024-04-14T15:00:29.993 INFO:teuthology.orchestra.run.smithi045.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-04-14T15:00:30.018 INFO:teuthology.orchestra.run.smithi045.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-04-14T15:00:30.060 INFO:teuthology.orchestra.run.smithi045.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-04-14T15:00:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-14T15:00:30.111 INFO:teuthology.orchestra.run.smithi045.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-14T15:00:30.136 INFO:teuthology.orchestra.run.smithi045.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 717 kB/s | 36 kB 00:00 2024-04-14T15:00:30.161 INFO:teuthology.orchestra.run.smithi045.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-14T15:00:30.186 INFO:teuthology.orchestra.run.smithi045.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-14T15:00:30.270 INFO:teuthology.orchestra.run.smithi045.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-14T15:00:30.337 INFO:teuthology.orchestra.run.smithi045.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.5 MB/s | 309 kB 00:00 2024-04-14T15:00:30.353 INFO:teuthology.orchestra.run.smithi045.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-04-14T15:00:30.387 INFO:teuthology.orchestra.run.smithi045.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-14T15:00:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-14T15:00:30.530 INFO:teuthology.orchestra.run.smithi045.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-14T15:00:30.705 INFO:teuthology.orchestra.run.smithi045.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-04-14T15:00:30.780 INFO:teuthology.orchestra.run.smithi045.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-14T15:00:30.972 INFO:teuthology.orchestra.run.smithi045.stdout:(31/32): librbd1-19.0.0-2778.ga971fd5f.el9.x86_ 12 MB/s | 3.1 MB 00:00 2024-04-14T15:00:31.147 INFO:teuthology.orchestra.run.smithi045.stdout:(32/32): librados2-19.0.0-2778.ga971fd5f.el9.x8 5.5 MB/s | 3.4 MB 00:00 2024-04-14T15:00:31.149 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:00:31.150 INFO:teuthology.orchestra.run.smithi045.stdout:Total 15 MB/s | 59 MB 00:03 2024-04-14T15:00:31.246 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:00:31.286 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:00:31.286 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:00:31.804 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:00:31.805 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:00:33.440 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 7.0 kB/s | 1.7 kB 00:00 2024-04-14T15:00:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:00:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-14T15:00:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-14T15:00:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-14T15:00:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-14T15:00:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-14T15:00:33.872 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-14T15:00:33.944 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-14T15:00:34.288 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-14T15:00:34.354 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:00:34.430 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:00:34.556 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 8/34 2024-04-14T15:00:34.580 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 9/34 2024-04-14T15:00:34.642 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 9/34 2024-04-14T15:00:34.672 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-14T15:00:34.707 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9. 11/34 2024-04-14T15:00:34.769 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 12/34 2024-04-14T15:00:34.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 13/34 2024-04-14T15:00:34.846 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 13/34 2024-04-14T15:00:34.881 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-14T15:00:34.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-14T15:00:35.047 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-14T15:00:35.090 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-14T15:00:35.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-14T15:00:35.159 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-14T15:00:35.192 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-14T15:00:35.531 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-14T15:00:35.563 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 22/34 2024-04-14T15:00:35.638 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 22/34 2024-04-14T15:00:35.668 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 23/34 2024-04-14T15:00:35.708 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.038 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.039 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:00:36.040 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.041 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.042 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.043 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.044 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.045 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.046 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:00:36.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-14T15:00:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 26/34 2024-04-14T15:00:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 26/34 2024-04-14T15:00:36.383 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 27/34 2024-04-14T15:00:36.418 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x8 28/34 2024-04-14T15:00:36.574 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:00:36.587 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:00:36.587 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:00:36.587 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 10 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.4 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.1 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 5.4 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 21 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 25 k 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-14T15:00:36.588 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 746 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 501 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 5.2 M 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 45 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 135 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 161 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-14T15:00:36.589 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 321 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 298 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 99 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:Install 30 Packages 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 2 Packages 2024-04-14T15:00:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:00:36.591 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 59 M 2024-04-14T15:00:36.591 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:00:38.658 INFO:teuthology.orchestra.run.smithi071.stdout:(1/32): ceph-base-19.0.0-2778.ga971fd5f.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-14T15:00:38.758 INFO:teuthology.orchestra.run.smithi071.stdout:(2/32): ceph-selinux-19.0.0-2778.ga971fd5f.el9. 252 kB/s | 25 kB 00:00 2024-04-14T15:00:38.785 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:38.831 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:38.950 INFO:teuthology.orchestra.run.smithi071.stdout:(3/32): ceph-common-19.0.0-2778.ga971fd5f.el9.x 11 MB/s | 21 MB 00:01 2024-04-14T15:00:39.050 INFO:teuthology.orchestra.run.smithi071.stdout:(4/32): libradosstriper1-19.0.0-2778.ga971fd5f. 4.9 MB/s | 501 kB 00:00 2024-04-14T15:00:39.108 INFO:teuthology.orchestra.run.smithi071.stdout:(5/32): libcephfs2-19.0.0-2778.ga971fd5f.el9.x8 2.1 MB/s | 746 kB 00:00 2024-04-14T15:00:39.217 INFO:teuthology.orchestra.run.smithi071.stdout:(6/32): python3-ceph-argparse-19.0.0-2778.ga971 416 kB/s | 45 kB 00:00 2024-04-14T15:00:39.326 INFO:teuthology.orchestra.run.smithi071.stdout:(7/32): python3-ceph-common-19.0.0-2778.ga971fd 1.2 MB/s | 135 kB 00:00 2024-04-14T15:00:39.414 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:39.451 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 30/34 2024-04-14T15:00:39.545 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 30/34 2024-04-14T15:00:39.546 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-14T15:00:39.546 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-14T15:00:39.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:39.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:00:39.601 INFO:teuthology.orchestra.run.smithi071.stdout:(8/32): librgw2-19.0.0-2778.ga971fd5f.el9.x86_6 9.5 MB/s | 5.2 MB 00:00 2024-04-14T15:00:39.627 INFO:teuthology.orchestra.run.smithi071.stdout:(9/32): python3-cephfs-19.0.0-2778.ga971fd5f.el 536 kB/s | 161 kB 00:00 2024-04-14T15:00:39.685 INFO:teuthology.orchestra.run.smithi071.stdout:(10/32): python3-rados-19.0.0-2778.ga971fd5f.el 3.8 MB/s | 321 kB 00:00 2024-04-14T15:00:39.752 INFO:teuthology.orchestra.run.smithi071.stdout:(11/32): python3-rbd-19.0.0-2778.ga971fd5f.el9. 2.3 MB/s | 298 kB 00:00 2024-04-14T15:00:39.777 INFO:teuthology.orchestra.run.smithi071.stdout:(12/32): python3-rgw-19.0.0-2778.ga971fd5f.el9. 1.1 MB/s | 99 kB 00:00 2024-04-14T15:00:39.978 INFO:teuthology.orchestra.run.smithi071.stdout:(13/32): ceph-radosgw-19.0.0-2778.ga971fd5f.el9 3.4 MB/s | 10 MB 00:02 2024-04-14T15:00:40.128 INFO:teuthology.orchestra.run.smithi071.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 285 kB/s | 107 kB 00:00 2024-04-14T15:00:40.162 INFO:teuthology.orchestra.run.smithi071.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 417 kB/s | 160 kB 00:00 2024-04-14T15:00:40.212 INFO:teuthology.orchestra.run.smithi071.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 194 kB/s | 45 kB 00:00 2024-04-14T15:00:40.312 INFO:teuthology.orchestra.run.smithi071.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-14T15:00:40.362 INFO:teuthology.orchestra.run.smithi071.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 281 kB/s | 42 kB 00:00 2024-04-14T15:00:40.463 INFO:teuthology.orchestra.run.smithi071.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 359 kB/s | 36 kB 00:00 2024-04-14T15:00:40.488 INFO:teuthology.orchestra.run.smithi071.stdout:(20/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-04-14T15:00:40.513 INFO:teuthology.orchestra.run.smithi071.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-04-14T15:00:40.539 INFO:teuthology.orchestra.run.smithi071.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-04-14T15:00:40.730 INFO:teuthology.orchestra.run.smithi071.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 185 kB/s | 26 kB 00:00 2024-04-14T15:00:40.789 INFO:teuthology.orchestra.run.smithi071.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-04-14T15:00:40.814 INFO:teuthology.orchestra.run.smithi071.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 947 kB/s | 309 kB 00:00 2024-04-14T15:00:40.848 INFO:teuthology.orchestra.run.smithi071.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-14T15:00:40.956 INFO:teuthology.orchestra.run.smithi071.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-14T15:00:41.149 INFO:teuthology.orchestra.run.smithi071.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-14T15:00:41.534 INFO:teuthology.orchestra.run.smithi071.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-04-14T15:00:42.207 INFO:teuthology.orchestra.run.smithi071.stdout:(30/32): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-14T15:00:42.266 INFO:teuthology.orchestra.run.smithi071.stdout:(31/32): librados2-19.0.0-2778.ga971fd5f.el9.x8 3.0 MB/s | 3.4 MB 00:01 2024-04-14T15:00:42.324 INFO:teuthology.orchestra.run.smithi071.stdout:(32/32): librbd1-19.0.0-2778.ga971fd5f.el9.x86_ 4.0 MB/s | 3.1 MB 00:00 2024-04-14T15:00:42.330 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:00:42.330 INFO:teuthology.orchestra.run.smithi071.stdout:Total 10 MB/s | 59 MB 00:05 2024-04-14T15:00:42.425 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:00:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:00:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:00:42.959 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:00:42.960 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:00:44.469 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:00:44.569 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-14T15:00:44.604 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-14T15:00:44.662 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-14T15:00:44.685 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-14T15:00:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-14T15:00:44.843 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-14T15:00:44.907 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-14T15:00:45.282 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-14T15:00:45.322 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:00:45.414 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:00:45.501 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 8/34 2024-04-14T15:00:45.532 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 9/34 2024-04-14T15:00:45.603 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 9/34 2024-04-14T15:00:45.634 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-14T15:00:45.674 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9. 11/34 2024-04-14T15:00:45.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 12/34 2024-04-14T15:00:45.767 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 13/34 2024-04-14T15:00:45.814 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 13/34 2024-04-14T15:00:45.849 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-14T15:00:45.957 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-14T15:00:45.992 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-14T15:00:46.032 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-14T15:00:46.057 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-14T15:00:46.086 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-14T15:00:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-14T15:00:46.430 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-14T15:00:46.456 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 22/34 2024-04-14T15:00:46.539 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 22/34 2024-04-14T15:00:46.569 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 23/34 2024-04-14T15:00:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-14T15:00:47.093 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-14T15:00:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 26/34 2024-04-14T15:00:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 26/34 2024-04-14T15:00:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 27/34 2024-04-14T15:00:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x8 28/34 2024-04-14T15:00:49.504 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:49.540 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-04-14T15:00:49.845 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-04-14T15:00:49.846 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-04-14T15:00:49.846 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-04-14T15:00:49.846 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:50.103 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 29/34 2024-04-14T15:00:50.152 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 30/34 2024-04-14T15:00:50.228 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 30/34 2024-04-14T15:00:50.228 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-14T15:00:50.228 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-14T15:00:50.228 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:00:50.270 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:00:50.800 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/34 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/34 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T15:00:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-14T15:00:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-14T15:00:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:00:52.077 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:00:52.077 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/34 2024-04-14T15:00:52.077 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 3/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 4/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 6/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9. 8/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x8 9/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 10/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 11/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 12/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 13/34 2024-04-14T15:00:52.078 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-14T15:00:52.079 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-14T15:00:52.080 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-14T15:00:52.080 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-14T15:00:52.080 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-14T15:00:52.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-14T15:00:52.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-14T15:00:52.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:00:52.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-14T15:00:52.081 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/34 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:00:53.122 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-14T15:00:53.123 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-04-14T15:00:53.124 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-14T15:00:53.125 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:53.125 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:00:53.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2024-04-14T15:00:54.040 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:30 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:00:54.343 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.344 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.345 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:00:54.346 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:00:54.347 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.348 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.349 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.350 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.351 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.352 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:00:54.417 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 49 M 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 163 k 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-14T15:00:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-04-14T15:00:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:00:54.420 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 49 M 2024-04-14T15:00:54.420 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 210 M 2024-04-14T15:00:54.420 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:00:54.937 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): libcephsqlite-19.0.0-2778.ga971fd5f.el9. 631 kB/s | 163 kB 00:00 2024-04-14T15:00:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): jq-1.6-16.el9.x86_64.rpm 659 kB/s | 187 kB 00:00 2024-04-14T15:00:55.037 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 218 kB 00:00 2024-04-14T15:00:55.129 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 943 kB/s | 64 kB 00:00 2024-04-14T15:00:55.221 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-14T15:00:57.222 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): ceph-test-19.0.0-2778.ga971fd5f.el9.x86_ 19 MB/s | 49 MB 00:02 2024-04-14T15:00:57.225 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:00:57.225 INFO:teuthology.orchestra.run.smithi045.stdout:Total 18 MB/s | 49 MB 00:02 2024-04-14T15:00:57.276 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:00:57.298 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:00:57.298 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:00:57.706 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:00:57.707 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:00:58.303 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:00:58.373 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-14T15:00:58.413 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-14T15:00:58.453 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-14T15:00:58.490 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-14T15:00:58.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/6 2024-04-14T15:00:59.870 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-04-14T15:00:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:00.822 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/34 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/34 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-14T15:01:00.847 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:00.850 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-14T15:01:00.933 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-14T15:01:00.934 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 3/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 4/34 2024-04-14T15:01:02.111 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_6 6/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 7/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9. 8/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x8 9/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 10/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 11/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 12/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 13/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-14T15:01:02.112 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-14T15:01:02.113 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-14T15:01:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 31/34 2024-04-14T15:01:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-14T15:01:02.114 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/34 2024-04-14T15:01:03.353 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/6 2024-04-14T15:01:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-14T15:01:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:03.354 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-04-14T15:01:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-14T15:01:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:03.357 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 6/6 2024-04-14T15:01:03.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-test 2024-04-14T15:01:04.296 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:31 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:04.584 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.585 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.586 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.587 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:04.588 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:04.589 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.590 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.591 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.592 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.593 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:04.656 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 49 M 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:01:04.657 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 163 k 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-04-14T15:01:04.658 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:04.659 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 49 M 2024-04-14T15:01:04.659 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 210 M 2024-04-14T15:01:04.659 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:05.462 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 772 kB/s | 187 kB 00:00 2024-04-14T15:01:05.562 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 218 kB 00:00 2024-04-14T15:01:05.646 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): libcephsqlite-19.0.0-2778.ga971fd5f.el9. 383 kB/s | 163 kB 00:00 2024-04-14T15:01:05.671 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-14T15:01:05.838 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-04-14T15:01:05.916 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 6/6 2024-04-14T15:01:05.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/6 2024-04-14T15:01:05.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/6 2024-04-14T15:01:05.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-14T15:01:05.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-14T15:01:05.917 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-14T15:01:06.265 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-16.el9.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:06.266 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:06.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2024-04-14T15:01:07.171 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:43 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.476 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.477 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.478 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.479 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:07.480 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.481 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.482 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.483 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.484 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:07.485 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.485 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.485 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.485 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:07.554 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 6.6 k 2024-04-14T15:01:07.557 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 2.4 M 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 1.5 M 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 249 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 4.7 M 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 17 M 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-14T15:01:07.558 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-14T15:01:07.559 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout:Install 36 Packages 2024-04-14T15:01:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:07.561 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 30 M 2024-04-14T15:01:07.561 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 108 M 2024-04-14T15:01:07.561 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:01:08.660 INFO:teuthology.orchestra.run.smithi045.stdout:(1/36): ceph-19.0.0-2778.ga971fd5f.el9.x86_64.r 41 kB/s | 6.6 kB 00:00 2024-04-14T15:01:09.360 INFO:teuthology.orchestra.run.smithi045.stdout:(2/36): ceph-mds-19.0.0-2778.ga971fd5f.el9.x86_ 2.7 MB/s | 2.4 MB 00:00 2024-04-14T15:01:09.727 INFO:teuthology.orchestra.run.smithi045.stdout:(3/36): ceph-mon-19.0.0-2778.ga971fd5f.el9.x86_ 4.4 MB/s | 4.7 MB 00:01 2024-04-14T15:01:09.777 INFO:teuthology.orchestra.run.smithi045.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2778.ga971 4.9 MB/s | 249 kB 00:00 2024-04-14T15:01:09.894 INFO:teuthology.orchestra.run.smithi045.stdout:(5/36): ceph-mgr-19.0.0-2778.ga971fd5f.el9.x86_ 1.1 MB/s | 1.5 MB 00:01 2024-04-14T15:01:09.995 INFO:teuthology.orchestra.run.smithi045.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-14T15:01:10.055 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): ceph-test-19.0.0-2778.ga971fd5f.el9.x86_ 10 MB/s | 49 MB 00:04 2024-04-14T15:01:10.057 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:10.057 INFO:teuthology.orchestra.run.smithi071.stdout:Total 9.2 MB/s | 49 MB 00:05 2024-04-14T15:01:10.062 INFO:teuthology.orchestra.run.smithi045.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-14T15:01:10.108 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:01:10.128 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:01:10.128 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:01:10.129 INFO:teuthology.orchestra.run.smithi045.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-04-14T15:01:10.196 INFO:teuthology.orchestra.run.smithi045.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.8 MB/s | 125 kB 00:00 2024-04-14T15:01:10.263 INFO:teuthology.orchestra.run.smithi045.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-04-14T15:01:10.338 INFO:teuthology.orchestra.run.smithi045.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 2.8 MB/s | 1.2 MB 00:00 2024-04-14T15:01:10.519 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:01:10.519 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:01:10.622 INFO:teuthology.orchestra.run.smithi045.stdout:(12/36): ceph-osd-19.0.0-2778.ga971fd5f.el9.x86 13 MB/s | 17 MB 00:01 2024-04-14T15:01:10.647 INFO:teuthology.orchestra.run.smithi045.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 491 kB/s | 188 kB 00:00 2024-04-14T15:01:10.672 INFO:teuthology.orchestra.run.smithi045.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 516 kB/s | 172 kB 00:00 2024-04-14T15:01:10.706 INFO:teuthology.orchestra.run.smithi045.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-04-14T15:01:10.731 INFO:teuthology.orchestra.run.smithi045.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-04-14T15:01:10.765 INFO:teuthology.orchestra.run.smithi045.stdout:(17/36): python3-markupsafe-1.1.1-12.el9.x86_64 244 kB/s | 35 kB 00:00 2024-04-14T15:01:10.798 INFO:teuthology.orchestra.run.smithi045.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-04-14T15:01:10.940 INFO:teuthology.orchestra.run.smithi045.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 248 kB/s | 43 kB 00:00 2024-04-14T15:01:11.016 INFO:teuthology.orchestra.run.smithi045.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 532 kB/s | 151 kB 00:00 2024-04-14T15:01:11.066 INFO:teuthology.orchestra.run.smithi045.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 643 kB/s | 172 kB 00:00 2024-04-14T15:01:11.091 INFO:teuthology.orchestra.run.smithi045.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-04-14T15:01:11.106 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:01:11.133 INFO:teuthology.orchestra.run.smithi045.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-04-14T15:01:11.167 INFO:teuthology.orchestra.run.smithi045.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-04-14T15:01:11.180 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-04-14T15:01:11.192 INFO:teuthology.orchestra.run.smithi045.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-04-14T15:01:11.217 INFO:teuthology.orchestra.run.smithi045.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-04-14T15:01:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-04-14T15:01:11.242 INFO:teuthology.orchestra.run.smithi045.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-04-14T15:01:11.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-04-14T15:01:11.268 INFO:teuthology.orchestra.run.smithi045.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-04-14T15:01:11.301 INFO:teuthology.orchestra.run.smithi045.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 939 kB/s | 79 kB 00:00 2024-04-14T15:01:11.305 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-04-14T15:01:11.332 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/6 2024-04-14T15:01:11.335 INFO:teuthology.orchestra.run.smithi045.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 246 kB/s | 16 kB 00:00 2024-04-14T15:01:11.369 INFO:teuthology.orchestra.run.smithi045.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-14T15:01:11.386 INFO:teuthology.orchestra.run.smithi045.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-04-14T15:01:11.402 INFO:teuthology.orchestra.run.smithi045.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-04-14T15:01:11.462 INFO:teuthology.orchestra.run.smithi045.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-14T15:01:11.478 INFO:teuthology.orchestra.run.smithi045.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 264 kB/s | 20 kB 00:00 2024-04-14T15:01:11.612 INFO:teuthology.orchestra.run.smithi045.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-04-14T15:01:11.613 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:11.614 INFO:teuthology.orchestra.run.smithi045.stdout:Total 7.4 MB/s | 30 MB 00:04 2024-04-14T15:01:11.755 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:01:11.797 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:01:11.797 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:01:12.232 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:01:12.232 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:01:13.608 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:01:13.665 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-14T15:01:13.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-14T15:01:13.734 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-14T15:01:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-14T15:01:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-14T15:01:13.891 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-14T15:01:13.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-14T15:01:14.072 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-14T15:01:14.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-14T15:01:14.154 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-14T15:01:14.187 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-14T15:01:14.227 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-14T15:01:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-14T15:01:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-14T15:01:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-14T15:01:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-14T15:01:14.425 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-14T15:01:14.501 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-14T15:01:14.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-14T15:01:14.719 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-14T15:01:14.788 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-14T15:01:14.826 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-14T15:01:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-14T15:01:14.948 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-14T15:01:15.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-14T15:01:15.189 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-14T15:01:15.270 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-14T15:01:15.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-14T15:01:15.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-14T15:01:15.534 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-14T15:01:15.702 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9. 31/36 2024-04-14T15:01:15.723 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/36 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/36 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T15:01:15.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/6 2024-04-14T15:01:15.980 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 6/6 2024-04-14T15:01:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/36 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/36 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T15:01:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 34/36 2024-04-14T15:01:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 34/36 2024-04-14T15:01:17.781 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:17.781 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-14T15:01:17.782 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T15:01:17.782 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T15:01:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:18.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 35/36 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 35/36 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T15:01:18.040 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:18.109 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 36/36 2024-04-14T15:01:18.278 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 6/6 2024-04-14T15:01:18.278 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/6 2024-04-14T15:01:18.278 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/6 2024-04-14T15:01:18.279 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2024-04-14T15:01:18.279 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2024-04-14T15:01:18.279 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-16.el9.x86_64 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:18.629 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-14T15:01:18.630 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-14T15:01:18.630 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-14T15:01:18.630 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:18.630 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:18.944 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph 2024-04-14T15:01:19.543 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:46 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:19.832 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:19.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:19.838 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.839 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.840 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.841 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:19.841 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.841 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.841 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.841 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:19.877 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 36/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 3/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 4/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9. 6/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-14T15:01:19.878 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-14T15:01:19.879 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-14T15:01:19.880 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-14T15:01:19.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-14T15:01:19.907 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:19.909 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:19.909 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:01:19.909 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:19.909 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 6.6 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 2.4 M 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 1.5 M 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 249 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 4.7 M 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 17 M 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-14T15:01:19.910 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-14T15:01:19.911 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:19.912 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout:Install 36 Packages 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 30 M 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 108 M 2024-04-14T15:01:19.913 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-14T15:01:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:20.805 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:01:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:20.805 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-14T15:01:20.806 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-14T15:01:20.807 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:20.808 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:20.923 INFO:teuthology.orchestra.run.smithi071.stdout:(1/36): ceph-19.0.0-2778.ga971fd5f.el9.x86_64.r 25 kB/s | 6.6 kB 00:00 2024-04-14T15:01:21.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2024-04-14T15:01:21.290 INFO:teuthology.orchestra.run.smithi071.stdout:(2/36): ceph-mgr-19.0.0-2778.ga971fd5f.el9.x86_ 2.4 MB/s | 1.5 MB 00:00 2024-04-14T15:01:21.466 INFO:teuthology.orchestra.run.smithi071.stdout:(3/36): ceph-mds-19.0.0-2778.ga971fd5f.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-04-14T15:01:21.532 INFO:teuthology.orchestra.run.smithi071.stdout:(4/36): ceph-mon-19.0.0-2778.ga971fd5f.el9.x86_ 7.7 MB/s | 4.7 MB 00:00 2024-04-14T15:01:21.576 INFO:teuthology.orchestra.run.smithi071.stdout:(5/36): ceph-mgr-modules-core-19.0.0-2778.ga971 2.2 MB/s | 249 kB 00:00 2024-04-14T15:01:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:57 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:22.093 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:22.093 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:22.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:22.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:22.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:22.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:22.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.100 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.101 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:22.102 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:01:22.166 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:22.167 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:01:22.168 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:22.255 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2024-04-14T15:01:22.532 INFO:teuthology.orchestra.run.smithi071.stdout:(6/36): ceph-osd-19.0.0-2778.ga971fd5f.el9.x86_ 14 MB/s | 17 MB 00:01 2024-04-14T15:01:22.557 INFO:teuthology.orchestra.run.smithi071.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 247 kB/s | 253 kB 00:01 2024-04-14T15:01:22.902 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:58 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:23.204 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:23.205 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:23.206 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.207 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.208 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.209 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.210 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.211 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.212 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.213 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.214 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:23.276 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 762 k 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:23.277 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:01:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:23.278 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 762 k 2024-04-14T15:01:23.278 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 768 k 2024-04-14T15:01:23.278 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:01:23.301 INFO:teuthology.orchestra.run.smithi071.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 139 kB/s | 106 kB 00:00 2024-04-14T15:01:23.368 INFO:teuthology.orchestra.run.smithi071.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 167 kB/s | 135 kB 00:00 2024-04-14T15:01:23.574 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-19.0.0-2778.ga971fd5f.el9.noarch.rpm 2.5 MB/s | 762 kB 00:00 2024-04-14T15:01:23.575 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:23.575 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.5 MB/s | 762 kB 00:00 2024-04-14T15:01:23.575 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:01:23.581 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:01:23.581 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:01:23.587 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:01:23.587 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:01:23.665 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:01:23.785 INFO:teuthology.orchestra.run.smithi071.stdout:(10/36): python3-cryptography-36.0.1-4.el9.x86_ 568 kB/s | 1.2 MB 00:02 2024-04-14T15:01:24.085 INFO:teuthology.orchestra.run.smithi071.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 159 kB/s | 125 kB 00:00 2024-04-14T15:01:24.126 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:24.215 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:24.277 INFO:teuthology.orchestra.run.smithi071.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 236 kB/s | 215 kB 00:00 2024-04-14T15:01:24.591 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:24.686 INFO:teuthology.orchestra.run.smithi071.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 209 kB/s | 188 kB 00:00 2024-04-14T15:01:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:24.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:24.886 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:01:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:24.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:24.886 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:24.919 INFO:teuthology.orchestra.run.smithi071.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 54 kB/s | 35 kB 00:00 2024-04-14T15:01:24.945 INFO:teuthology.orchestra.run.smithi071.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 200 kB/s | 172 kB 00:00 2024-04-14T15:01:25.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2024-04-14T15:01:25.312 INFO:teuthology.orchestra.run.smithi071.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-04-14T15:01:25.503 INFO:teuthology.orchestra.run.smithi071.stdout:(17/36): python3-pytz-2021.1-5.el9.noarch.rpm 62 kB/s | 51 kB 00:00 2024-04-14T15:01:25.737 INFO:teuthology.orchestra.run.smithi071.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-14T15:01:25.750 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:01 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:25.762 INFO:teuthology.orchestra.run.smithi071.stdout:(19/36): python3-toml-0.10.2-6.el9.noarch.rpm 49 kB/s | 42 kB 00:00 2024-04-14T15:01:25.812 INFO:teuthology.orchestra.run.smithi071.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 141 kB/s | 43 kB 00:00 2024-04-14T15:01:25.913 INFO:teuthology.orchestra.run.smithi071.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-04-14T15:01:25.971 INFO:teuthology.orchestra.run.smithi071.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 736 kB/s | 172 kB 00:00 2024-04-14T15:01:25.996 INFO:teuthology.orchestra.run.smithi071.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 212 kB/s | 18 kB 00:00 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.054 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.055 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.056 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.057 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.058 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.059 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.060 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.061 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.062 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:26.063 INFO:teuthology.orchestra.run.smithi071.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 278 kB/s | 23 kB 00:00 2024-04-14T15:01:26.089 INFO:teuthology.orchestra.run.smithi071.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-04-14T15:01:26.128 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:26.128 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 145 k 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 145 k 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 435 k 2024-04-14T15:01:26.129 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:01:26.147 INFO:teuthology.orchestra.run.smithi071.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-04-14T15:01:26.173 INFO:teuthology.orchestra.run.smithi071.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 554 kB/s | 46 kB 00:00 2024-04-14T15:01:26.215 INFO:teuthology.orchestra.run.smithi071.stdout:(28/36): python3-cherrypy-18.6.1-2.el9.noarch.r 793 kB/s | 358 kB 00:00 2024-04-14T15:01:26.240 INFO:teuthology.orchestra.run.smithi071.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 855 kB/s | 79 kB 00:00 2024-04-14T15:01:26.282 INFO:teuthology.orchestra.run.smithi071.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 245 kB/s | 16 kB 00:00 2024-04-14T15:01:26.307 INFO:teuthology.orchestra.run.smithi071.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-14T15:01:26.349 INFO:teuthology.orchestra.run.smithi071.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 535 kB/s | 36 kB 00:00 2024-04-14T15:01:26.349 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-19.0.0-2778.ga971fd 658 kB/s | 145 kB 00:00 2024-04-14T15:01:26.350 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:26.350 INFO:teuthology.orchestra.run.smithi045.stdout:Total 654 kB/s | 145 kB 00:00 2024-04-14T15:01:26.350 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:01:26.359 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:01:26.359 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:01:26.382 INFO:teuthology.orchestra.run.smithi071.stdout:(33/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 632 kB/s | 90 kB 00:00 2024-04-14T15:01:26.408 INFO:teuthology.orchestra.run.smithi071.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-14T15:01:26.420 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:01:26.420 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:01:26.449 INFO:teuthology.orchestra.run.smithi071.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-04-14T15:01:26.475 INFO:teuthology.orchestra.run.smithi071.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-04-14T15:01:26.477 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:26.477 INFO:teuthology.orchestra.run.smithi071.stdout:Total 4.6 MB/s | 30 MB 00:06 2024-04-14T15:01:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:01:26.616 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:01:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:26.655 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:01:26.655 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:01:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:26.667 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:26.667 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-14T15:01:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:27.052 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:01:27.053 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:27.677 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:27.958 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2024-04-14T15:01:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:01:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-14T15:01:28.480 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-14T15:01:28.513 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-14T15:01:28.567 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:04 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:28.583 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-14T15:01:28.620 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-14T15:01:28.653 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-14T15:01:28.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-14T15:01:28.814 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-14T15:01:28.855 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:28.870 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:28.871 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.872 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.873 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.874 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.875 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.876 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.877 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:28.878 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:01:28.890 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-14T15:01:28.920 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-14T15:01:28.942 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:28.943 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:01:28.943 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:28.954 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-14T15:01:28.979 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-14T15:01:29.020 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-14T15:01:29.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2024-04-14T15:01:29.054 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-14T15:01:29.089 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-14T15:01:29.128 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-14T15:01:29.199 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-14T15:01:29.335 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-14T15:01:29.409 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-14T15:01:29.470 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-14T15:01:29.510 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-14T15:01:29.558 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-14T15:01:29.613 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-14T15:01:29.629 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:05 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:29.797 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-14T15:01:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-14T15:01:29.916 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.932 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:29.933 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.934 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.935 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:29.936 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:29.937 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.938 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.939 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.940 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:29.941 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:30.004 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-14T15:01:30.005 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 3.5 M 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 34 k 2024-04-14T15:01:30.007 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 16 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-04-14T15:01:30.008 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:30.009 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 8.6 M 2024-04-14T15:01:30.009 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 104 M 2024-04-14T15:01:30.009 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:01:30.046 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-14T15:01:30.161 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-14T15:01:30.338 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9. 31/36 2024-04-14T15:01:30.364 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/36 2024-04-14T15:01:30.389 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 32/36 2024-04-14T15:01:30.389 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:30.389 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-14T15:01:30.389 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T15:01:30.390 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-14T15:01:30.390 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:30.912 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2778.ga97 83 kB/s | 16 kB 00:00 2024-04-14T15:01:31.013 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2778.ga9 115 kB/s | 34 kB 00:00 2024-04-14T15:01:31.272 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): ceph-mgr-dashboard-19.0.0-2778.ga971fd5 6.4 MB/s | 3.5 MB 00:00 2024-04-14T15:01:31.372 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 8.4 MB/s | 863 kB 00:00 2024-04-14T15:01:31.397 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): python3-protobuf-3.14.0-13.el9.noarch.r 701 kB/s | 269 kB 00:00 2024-04-14T15:01:31.464 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 293 kB/s | 19 kB 00:00 2024-04-14T15:01:31.548 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.0 MB/s | 548 kB 00:00 2024-04-14T15:01:31.590 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-04-14T15:01:31.623 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 927 kB/s | 31 kB 00:00 2024-04-14T15:01:31.665 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-04-14T15:01:31.691 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2024-04-14T15:01:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/36 2024-04-14T15:01:31.874 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-04-14T15:01:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 33/36 2024-04-14T15:01:31.875 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:31.875 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-14T15:01:31.875 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T15:01:31.876 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-14T15:01:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:31.877 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:31.877 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.6 MB/s | 8.6 MB 00:01 2024-04-14T15:01:31.927 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:01:31.947 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:01:31.947 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:01:32.141 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:01:32.142 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:01:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 34/36 2024-04-14T15:01:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 34/36 2024-04-14T15:01:32.359 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:32.359 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-14T15:01:32.359 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T15:01:32.359 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-14T15:01:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 35/36 2024-04-14T15:01:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 35/36 2024-04-14T15:01:32.613 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:32.613 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-14T15:01:32.613 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T15:01:32.613 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-14T15:01:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 36/36 2024-04-14T15:01:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:01:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-14T15:01:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-14T15:01:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-14T15:01:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-14T15:01:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-14T15:01:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-14T15:01:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-14T15:01:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-14T15:01:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-14T15:01:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9 10/12 2024-04-14T15:01:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 36/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 3/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 4/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 5/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9. 6/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-14T15:01:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-14T15:01:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-14T15:01:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-14T15:01:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-14T15:01:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-14T15:01:34.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el 11/12 2024-04-14T15:01:34.504 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 12/12 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:35.466 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-14T15:01:35.467 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-14T15:01:35.468 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-14T15:01:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-14T15:01:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-14T15:01:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-14T15:01:35.469 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:35.469 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 12/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el 1/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 2/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9 3/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-14T15:01:35.834 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-14T15:01:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-14T15:01:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-14T15:01:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-14T15:01:35.835 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-14T15:01:35.844 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-base 2024-04-14T15:01:36.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-14T15:01:36.200 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-14T15:01:36.201 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-14T15:01:36.201 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:36.201 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:01:36.433 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:03 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:36.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.720 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.721 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.722 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.723 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.724 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.725 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.726 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.727 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.728 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:36.729 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-base-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:01:36.788 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:36.789 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:01:36.790 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:36.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install cephadm 2024-04-14T15:01:37.059 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:13 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:01:37.364 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:01:37.364 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.365 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.366 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:01:37.367 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.368 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.369 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:01:37.370 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.371 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.372 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.373 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.440 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 7.4 M 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-14T15:01:37.442 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-04-14T15:01:37.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 38 M 2024-04-14T15:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 209 M 2024-04-14T15:01:37.444 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:01:37.457 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:04 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.750 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:37.751 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:37.752 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.753 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.754 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:37.755 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.756 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.757 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.758 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:37.818 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 762 k 2024-04-14T15:01:37.819 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 762 k 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 768 k 2024-04-14T15:01:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:38.280 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm-19.0.0-2778.ga971fd5f.el9.noarch.rpm 1.6 MB/s | 762 kB 00:00 2024-04-14T15:01:38.280 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:38.280 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.6 MB/s | 762 kB 00:00 2024-04-14T15:01:38.281 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:01:38.286 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:01:38.286 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:01:38.292 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:01:38.292 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:01:38.371 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:01:38.772 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:38.850 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:39.214 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:39.540 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/1 2024-04-14T15:01:39.540 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:39.540 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:39.540 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:39.541 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:39.541 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:39.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-immutable-object-cache 2024-04-14T15:01:40.352 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:07 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.642 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:40.643 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:40.644 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:40.645 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.646 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.647 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.648 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.649 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.650 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.651 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.651 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:40.713 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:40.713 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 145 k 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 145 k 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 435 k 2024-04-14T15:01:40.714 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:40.943 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-immutable-object-cache-19.0.0-2778.ga971fd 632 kB/s | 145 kB 00:00 2024-04-14T15:01:40.944 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:40.944 INFO:teuthology.orchestra.run.smithi071.stdout:Total 629 kB/s | 145 kB 00:00 2024-04-14T15:01:40.944 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:01:40.951 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:01:40.952 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:01:41.008 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:01:41.009 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:01:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:01:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:41.243 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:01:41.243 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-14T15:01:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:41.551 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 776 kB/s | 195 kB 00:00 2024-04-14T15:01:41.635 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-14T15:01:42.143 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-27 8.8 MB/s | 7.4 MB 00:00 2024-04-14T15:01:42.210 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f. 1/1 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:42.249 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:42.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr 2024-04-14T15:01:42.702 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 33 kB/s | 18 kB 00:00 2024-04-14T15:01:42.844 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 56 kB/s | 35 kB 00:00 2024-04-14T15:01:43.103 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:10 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.394 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:43.395 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:43.396 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.397 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.398 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.399 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.400 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:43.401 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.402 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:43.403 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.403 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.403 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.403 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:43.403 INFO:teuthology.orchestra.run.smithi071.stdout:Package ceph-mgr-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:01:43.462 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:43.463 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:01:43.463 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:43.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-dashboard 2024-04-14T15:01:43.777 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 263 kB/s | 245 kB 00:00 2024-04-14T15:01:44.127 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:11 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:44.418 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:44.419 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:44.420 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:44.421 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:44.422 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:44.423 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.424 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:44.425 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.426 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:44.427 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.427 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.427 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.427 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:44.487 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 3.5 M 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 34 k 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 16 k 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-14T15:01:44.489 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout:Install 12 Packages 2024-04-14T15:01:44.490 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:44.491 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 8.6 M 2024-04-14T15:01:44.491 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 104 M 2024-04-14T15:01:44.491 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:45.229 INFO:teuthology.orchestra.run.smithi071.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2778.ga97 80 kB/s | 16 kB 00:00 2024-04-14T15:01:45.330 INFO:teuthology.orchestra.run.smithi071.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2778.ga9 111 kB/s | 34 kB 00:00 2024-04-14T15:01:45.413 INFO:teuthology.orchestra.run.smithi071.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-04-14T15:01:45.480 INFO:teuthology.orchestra.run.smithi071.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.8 MB/s | 269 kB 00:00 2024-04-14T15:01:45.530 INFO:teuthology.orchestra.run.smithi071.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2778.ga971fd5 7.0 MB/s | 3.5 MB 00:00 2024-04-14T15:01:45.964 INFO:teuthology.orchestra.run.smithi071.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 45 kB/s | 19 kB 00:00 2024-04-14T15:01:46.219 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 668 kB/s | 3.0 MB 00:04 2024-04-14T15:01:46.473 INFO:teuthology.orchestra.run.smithi071.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 552 kB/s | 548 kB 00:00 2024-04-14T15:01:46.498 INFO:teuthology.orchestra.run.smithi071.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 796 kB/s | 863 kB 00:01 2024-04-14T15:01:46.632 INFO:teuthology.orchestra.run.smithi071.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 911 kB/s | 144 kB 00:00 2024-04-14T15:01:46.790 INFO:teuthology.orchestra.run.smithi071.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-04-14T15:01:46.923 INFO:teuthology.orchestra.run.smithi071.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 72 kB/s | 31 kB 00:00 2024-04-14T15:01:47.099 INFO:teuthology.orchestra.run.smithi071.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 1.8 MB/s | 2.0 MB 00:01 2024-04-14T15:01:47.100 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:01:47.100 INFO:teuthology.orchestra.run.smithi071.stdout:Total 3.3 MB/s | 8.6 MB 00:02 2024-04-14T15:01:47.144 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 271 kB/s | 250 kB 00:00 2024-04-14T15:01:47.151 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:01:47.170 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:01:47.170 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:01:47.354 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:01:47.354 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:01:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:01:47.966 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-14T15:01:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-14T15:01:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-14T15:01:48.095 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-14T15:01:48.210 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-14T15:01:48.336 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-14T15:01:48.431 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-14T15:01:48.474 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-14T15:01:48.502 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-14T15:01:48.535 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9 10/12 2024-04-14T15:01:49.328 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 732 kB/s | 4.7 MB 00:06 2024-04-14T15:01:49.570 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el 11/12 2024-04-14T15:01:49.610 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 12/12 2024-04-14T15:01:50.778 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 746 kB/s | 5.1 MB 00:07 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 12/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el 1/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noa 2/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9 3/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-14T15:01:50.973 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-14T15:01:50.974 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:01:51.338 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:51.339 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:01:51.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-14T15:01:52.170 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:19 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:01:52.457 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:01:52.457 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.458 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.459 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:01:52.460 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.461 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:01:52.462 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.463 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.464 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.465 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.466 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:01:52.467 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.467 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.467 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.467 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:01:52.525 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:01:52.526 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 7.4 M 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-14T15:01:52.527 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout:Install 12 Packages 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:01:52.528 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 38 M 2024-04-14T15:01:52.529 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 209 M 2024-04-14T15:01:52.529 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:01:54.589 INFO:teuthology.orchestra.run.smithi071.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-14T15:01:54.656 INFO:teuthology.orchestra.run.smithi071.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-14T15:01:54.798 INFO:teuthology.orchestra.run.smithi071.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 157 kB/s | 33 kB 00:00 2024-04-14T15:01:54.881 INFO:teuthology.orchestra.run.smithi071.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-04-14T15:01:54.965 INFO:teuthology.orchestra.run.smithi071.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-14T15:01:55.273 INFO:teuthology.orchestra.run.smithi071.stdout:(6/12): ceph-mgr-diskprediction-local-19.0.0-27 8.6 MB/s | 7.4 MB 00:00 2024-04-14T15:01:56.032 INFO:teuthology.orchestra.run.smithi071.stdout:(7/12): python3-devel-3.9.17-2.el9.x86_64.rpm 324 kB/s | 245 kB 00:00 2024-04-14T15:02:07.688 INFO:teuthology.orchestra.run.smithi071.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 235 kB/s | 3.0 MB 00:13 2024-04-14T15:02:08.688 INFO:teuthology.orchestra.run.smithi071.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 354 kB/s | 4.7 MB 00:13 2024-04-14T15:02:08.701 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 772 kB/s | 16 MB 00:21 2024-04-14T15:02:08.702 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:08.702 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.2 MB/s | 38 MB 00:31 2024-04-14T15:02:08.713 INFO:teuthology.orchestra.run.smithi071.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 244 kB/s | 250 kB 00:01 2024-04-14T15:02:09.063 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:09.090 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:09.090 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:09.545 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:09.545 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:09.772 INFO:teuthology.orchestra.run.smithi071.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 380 kB/s | 5.1 MB 00:13 2024-04-14T15:02:10.327 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:10.396 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-14T15:02:10.441 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-14T15:02:10.554 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-14T15:02:10.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-14T15:02:10.776 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-14T15:02:10.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-14T15:02:10.911 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-14T15:02:11.592 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-14T15:02:11.716 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-14T15:02:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-14T15:02:14.104 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-14T15:02:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 12/12 2024-04-14T15:02:16.597 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 12/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 1/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-14T15:02:16.598 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-14T15:02:17.022 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:17.023 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:17.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2024-04-14T15:02:18.001 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:54 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:18.309 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:18.310 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:18.311 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:18.312 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.313 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.314 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.315 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.316 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:18.317 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.317 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.317 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.317 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:18.381 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 49 k 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-14T15:02:18.383 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2024-04-14T15:02:18.384 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:18.385 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.2 M 2024-04-14T15:02:18.385 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2024-04-14T15:02:18.385 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:18.768 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 165 kB/s | 19 kB 00:00 2024-04-14T15:02:18.794 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 26 kB 00:00 2024-04-14T15:02:18.827 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): ceph-mgr-rook-19.0.0-2778.ga971fd5f.el9 282 kB/s | 49 kB 00:00 2024-04-14T15:02:18.878 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-04-14T15:02:18.903 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-14T15:02:18.928 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-14T15:02:19.020 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 279 kB 00:00 2024-04-14T15:02:19.062 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2024-04-14T15:02:19.112 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 81 kB/s | 15 kB 00:00 2024-04-14T15:02:19.180 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 898 kB/s | 59 kB 00:00 2024-04-14T15:02:19.238 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-04-14T15:02:19.263 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-google-auth-2.28.2-1.el9.noarc 839 kB/s | 203 kB 00:00 2024-04-14T15:02:19.288 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 825 kB/s | 90 kB 00:00 2024-04-14T15:02:19.290 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:19.291 INFO:teuthology.orchestra.run.smithi045.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-04-14T15:02:19.354 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:19.369 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:19.369 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:19.470 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:19.471 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:19.996 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:20.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-14T15:02:20.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-14T15:02:20.291 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-14T15:02:20.327 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-14T15:02:20.369 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-14T15:02:20.475 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-14T15:02:20.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-14T15:02:20.646 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-14T15:02:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.4 MB/s | 16 MB 00:11 2024-04-14T15:02:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:20.680 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.4 MB/s | 38 MB 00:28 2024-04-14T15:02:21.033 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:21.061 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:21.061 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:21.407 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-14T15:02:21.459 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-14T15:02:21.491 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:21.491 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:21.493 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-14T15:02:21.542 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-14T15:02:21.571 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 13/13 2024-04-14T15:02:22.272 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 13/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 1/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-14T15:02:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-14T15:02:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-14T15:02:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-14T15:02:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-14T15:02:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-14T15:02:22.375 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-14T15:02:22.418 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-14T15:02:22.517 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-14T15:02:22.552 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-04-14T15:02:22.730 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-14T15:02:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-14T15:02:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:22.746 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:22.747 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:22.760 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-14T15:02:22.836 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-14T15:02:23.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2024-04-14T15:02:23.493 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-14T15:02:23.605 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-14T15:02:23.610 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:59 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:23.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:23.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:23.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:23.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:23.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:23.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:23.985 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 141 k 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:02:23.986 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-04-14T15:02:23.987 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:23.988 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 7.0 M 2024-04-14T15:02:23.988 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2024-04-14T15:02:23.988 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:24.711 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2778.ga971fd5f.e 661 kB/s | 141 kB 00:00 2024-04-14T15:02:24.736 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-04-14T15:02:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-14T15:02:24.836 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 577 kB/s | 58 kB 00:00 2024-04-14T15:02:24.887 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-04-14T15:02:24.912 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.7 MB/s | 548 kB 00:00 2024-04-14T15:02:25.788 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-04-14T15:02:25.788 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:25.788 INFO:teuthology.orchestra.run.smithi045.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-04-14T15:02:25.862 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:25.865 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-14T15:02:25.874 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:25.874 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:25.891 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 12/12 2024-04-14T15:02:25.981 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:25.982 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:26.304 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-14T15:02:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-14T15:02:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-14T15:02:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-14T15:02:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-14T15:02:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 6/6 2024-04-14T15:02:28.043 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 6/6 2024-04-14T15:02:28.043 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/6 2024-04-14T15:02:28.043 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-14T15:02:28.043 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-14T15:02:28.043 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-14T15:02:28.044 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-14T15:02:28.357 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-14T15:02:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-14T15:02:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-14T15:02:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-14T15:02:28.358 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:28.358 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 12/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971f 1/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-14T15:02:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-14T15:02:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-14T15:02:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-14T15:02:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 9/12 2024-04-14T15:02:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-14T15:02:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-14T15:02:28.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-14T15:02:28.836 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:28.837 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:29.229 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:05 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:29.287 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-rook 2024-04-14T15:02:29.533 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:29.533 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:29.533 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:29.534 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:29.538 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.539 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:29.540 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:29.541 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.542 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.542 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.542 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:29.607 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 891 k 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:29.608 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 971 k 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.8 M 2024-04-14T15:02:29.609 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:29.895 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:01:56 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:29.989 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 745 kB/s | 80 kB 00:00 2024-04-14T15:02:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.183 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:30.184 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:30.185 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:30.186 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.187 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.188 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.189 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.190 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:30.191 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-fuse-19.0.0-2778.ga971fd5f.el9.x86_ 2.8 MB/s | 891 kB 00:00 2024-04-14T15:02:30.192 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:30.192 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.6 MB/s | 971 kB 00:00 2024-04-14T15:02:30.202 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:30.224 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:30.224 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:30.253 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 49 k 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-14T15:02:30.254 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-04-14T15:02:30.255 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:30.256 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 2.2 M 2024-04-14T15:02:30.256 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 26 M 2024-04-14T15:02:30.256 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:30.305 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:30.306 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:30.512 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:30.640 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 271 kB/s | 26 kB 00:00 2024-04-14T15:02:30.654 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-14T15:02:30.666 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 160 kB/s | 19 kB 00:00 2024-04-14T15:02:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-04-14T15:02:30.726 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/2 2024-04-14T15:02:30.749 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): ceph-mgr-rook-19.0.0-2778.ga971fd5f.el9 241 kB/s | 49 kB 00:00 2024-04-14T15:02:30.774 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-14T15:02:30.800 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-04-14T15:02:30.875 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 428 kB/s | 54 kB 00:00 2024-04-14T15:02:31.067 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 110 kB/s | 32 kB 00:00 2024-04-14T15:02:31.092 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 51 kB/s | 15 kB 00:00 2024-04-14T15:02:31.301 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 476 kB/s | 203 kB 00:00 2024-04-14T15:02:31.349 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/2 2024-04-14T15:02:31.349 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/2 2024-04-14T15:02:31.443 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-14T15:02:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-14T15:02:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:31.677 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-14T15:02:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:31.678 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:31.726 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 211 kB/s | 90 kB 00:00 2024-04-14T15:02:31.760 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-04-14T15:02:31.761 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:31.761 INFO:teuthology.orchestra.run.smithi071.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-04-14T15:02:31.831 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:31.847 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:31.847 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:31.943 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:31.943 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:31.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2024-04-14T15:02:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:32.612 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:08 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-14T15:02:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-14T15:02:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-14T15:02:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-14T15:02:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-14T15:02:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:32.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.919 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:32.920 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.921 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:32.922 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.923 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.924 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.925 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:32.992 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 264 k 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-04-14T15:02:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:32.994 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 341 k 2024-04-14T15:02:32.994 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.5 M 2024-04-14T15:02:32.994 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-14T15:02:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-14T15:02:33.413 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 678 kB/s | 77 kB 00:00 2024-04-14T15:02:33.538 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): ceph-volume-19.0.0-2778.ga971fd5f.el9.no 1.1 MB/s | 264 kB 00:00 2024-04-14T15:02:33.539 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:33.539 INFO:teuthology.orchestra.run.smithi045.stdout:Total 624 kB/s | 341 kB 00:00 2024-04-14T15:02:33.551 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:33.562 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:33.562 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:33.607 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:33.608 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:33.762 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-14T15:02:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-14T15:02:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-14T15:02:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-14T15:02:33.930 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 13/13 2024-04-14T15:02:33.935 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-14T15:02:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2/2 2024-04-14T15:02:33.981 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2/2 2024-04-14T15:02:33.981 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:02:33.981 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-14T15:02:33.981 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:34.677 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 1/2 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 13/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 1/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-14T15:02:34.746 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-14T15:02:34.747 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-14T15:02:34.747 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-14T15:02:34.747 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-14T15:02:34.747 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-14T15:02:35.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-14T15:02:35.061 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:35.061 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:35.061 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:35.062 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-14T15:02:35.062 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:35.062 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:35.138 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-14T15:02:35.138 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:35.138 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:35.138 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:35.138 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:35.139 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:35.398 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-mgr-cephadm 2024-04-14T15:02:35.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2024-04-14T15:02:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:02 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:36.053 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:12 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:36.286 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:36.286 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:36.286 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:36.286 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.287 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.288 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:36.289 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:36.290 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.291 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.292 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:36.293 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.294 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.356 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.357 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:36.358 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:36.359 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:36.360 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.361 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.362 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.363 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.364 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 141 k 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:02:36.365 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 7.0 M 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 33 M 2024-04-14T15:02:36.366 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:36.428 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-04-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:36.429 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 127 k 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 127 k 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 456 k 2024-04-14T15:02:36.430 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-19.0.0-2778.ga971fd5f.el9.x86_64 403 kB/s | 127 kB 00:00 2024-04-14T15:02:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:Total 401 kB/s | 127 kB 00:00 2024-04-14T15:02:36.746 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:36.751 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:36.752 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:36.814 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:36.819 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2778.ga971fd5f.e 655 kB/s | 141 kB 00:00 2024-04-14T15:02:36.969 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:37.065 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:37.186 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.5 MB/s | 548 kB 00:00 2024-04-14T15:02:37.229 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 396 kB/s | 248 kB 00:00 2024-04-14T15:02:37.262 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-04-14T15:02:37.445 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 236 kB/s | 51 kB 00:00 2024-04-14T15:02:37.498 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:38.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2024-04-14T15:02:38.705 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:14 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:39.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:39.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:39.014 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:39.015 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.016 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:39.017 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:39.081 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:39.082 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:39.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2024-04-14T15:02:39.404 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.1 MB/s | 6.0 MB 00:02 2024-04-14T15:02:39.404 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:39.405 INFO:teuthology.orchestra.run.smithi071.stdout:Total 2.3 MB/s | 7.0 MB 00:03 2024-04-14T15:02:39.477 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:39.488 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:39.488 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:39.592 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:39.593 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:39.779 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:15 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:39.905 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:40.068 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-14T15:02:40.081 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:40.081 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:40.082 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:40.083 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:40.084 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.085 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.086 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.087 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.088 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.089 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.090 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:40.129 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-14T15:02:40.154 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 32 k 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:02:40.155 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:40.156 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 32 k 2024-04-14T15:02:40.156 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 142 k 2024-04-14T15:02:40.156 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:40.558 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-19.0.0-2778.ga971fd5f.el9.x86_6 79 kB/s | 32 kB 00:00 2024-04-14T15:02:40.558 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:40.558 INFO:teuthology.orchestra.run.smithi045.stdout:Total 79 kB/s | 32 kB 00:00 2024-04-14T15:02:40.559 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:40.562 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:40.562 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:40.589 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:40.589 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:40.644 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-14T15:02:40.699 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-14T15:02:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-14T15:02:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 6/6 2024-04-14T15:02:41.158 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:41.438 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:41.439 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 6/6 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 1/6 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-14T15:02:41.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-14T15:02:41.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2024-04-14T15:02:41.936 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-14T15:02:41.936 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:41.937 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:42.155 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-fuse 2024-04-14T15:02:42.248 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:18 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.552 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:42.553 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.554 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.555 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:42.556 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:42.557 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.558 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.559 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:42.560 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:42.628 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:42.629 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:42.629 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:42.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2024-04-14T15:02:42.756 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:09 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:43.052 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:43.053 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:43.054 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.055 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.056 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.057 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.058 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:43.059 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.060 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.123 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:43.123 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 891 k 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout:Install 2 Packages 2024-04-14T15:02:43.124 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:43.125 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 971 k 2024-04-14T15:02:43.125 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 2.8 M 2024-04-14T15:02:43.125 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:43.333 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:19 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:43.636 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.637 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:43.638 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:43.639 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:43.640 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.641 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.642 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:43.643 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.644 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.644 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.644 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:43.644 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:43.710 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:43.711 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:43.711 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:43.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2024-04-14T15:02:44.412 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:20 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:44.720 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:44.721 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:44.722 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:44.723 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.724 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:44.725 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.726 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.727 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:44.728 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:44.794 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:44.795 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:44.795 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:44.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2024-04-14T15:02:45.506 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:21 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:45.814 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:45.815 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.816 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:45.817 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:45.818 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.819 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:45.820 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:45.821 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.822 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:45.823 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:45.887 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:45.888 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:45.889 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:45.970 INFO:teuthology.orchestra.run.smithi071.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-04-14T15:02:45.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2024-04-14T15:02:46.606 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:22 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:46.910 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:46.910 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:46.911 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:46.912 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.913 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:46.914 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.915 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.916 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.917 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.918 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:46.919 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:46.984 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:46.986 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:46.986 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:47.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2024-04-14T15:02:47.337 INFO:teuthology.orchestra.run.smithi071.stdout:(2/2): ceph-fuse-19.0.0-2778.ga971fd5f.el9.x86_ 616 kB/s | 891 kB 00:01 2024-04-14T15:02:47.337 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:47.338 INFO:teuthology.orchestra.run.smithi071.stdout:Total 230 kB/s | 971 kB 00:04 2024-04-14T15:02:47.352 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:47.376 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:47.377 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:47.458 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:47.458 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:47.639 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:47.701 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:23 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:47.780 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-14T15:02:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/2 2024-04-14T15:02:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.005 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.006 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.007 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:48.008 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.009 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.010 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:48.011 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.012 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.013 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:48.013 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:48.078 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:48.079 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-14T15:02:48.079 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:48.172 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2024-04-14T15:02:48.608 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2/2 2024-04-14T15:02:48.608 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/2 2024-04-14T15:02:48.784 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:24 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-14T15:02:48.928 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:48.928 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-14T15:02:48.929 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:48.929 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:49.090 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:49.091 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.092 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:49.093 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.094 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.095 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.096 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.097 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.098 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.099 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 85 k 2024-04-14T15:02:49.165 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 85 k 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 231 k 2024-04-14T15:02:49.166 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:49.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-volume 2024-04-14T15:02:49.378 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 403 kB/s | 85 kB 00:00 2024-04-14T15:02:49.379 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:49.379 INFO:teuthology.orchestra.run.smithi045.stdout:Total 400 kB/s | 85 kB 00:00 2024-04-14T15:02:49.379 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:49.387 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:49.387 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:49.449 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:49.449 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:49.653 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:49.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:49.848 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:16 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:50.140 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.141 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:50.142 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.143 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.144 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.145 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:50.146 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:50.147 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.148 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:50.149 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:50.209 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-2778.ga971fd5f.el9 ceph-noarch 264 k 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:50.210 INFO:teuthology.orchestra.run.smithi071.stdout:Install 2 Packages 2024-04-14T15:02:50.211 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:50.211 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 341 k 2024-04-14T15:02:50.211 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 1.5 M 2024-04-14T15:02:50.211 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:50.454 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:50.454 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:50.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2024-04-14T15:02:50.691 INFO:teuthology.orchestra.run.smithi071.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 232 kB/s | 77 kB 00:00 2024-04-14T15:02:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:(2/2): ceph-volume-19.0.0-2778.ga971fd5f.el9.no 555 kB/s | 264 kB 00:00 2024-04-14T15:02:50.833 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:50.834 INFO:teuthology.orchestra.run.smithi071.stdout:Total 547 kB/s | 341 kB 00:00 2024-04-14T15:02:50.845 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:50.855 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:50.855 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:50.894 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:50.894 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:51.047 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:51.223 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-14T15:02:51.261 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2/2 2024-04-14T15:02:51.276 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2/2 2024-04-14T15:02:51.276 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:02:51.276 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-14T15:02:51.276 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:51.291 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:27 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.595 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:51.596 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:51.597 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.598 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:51.599 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.600 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:51.601 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.602 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.603 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:51.667 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.1 M 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:51.668 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:02:51.669 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.1 M 2024-04-14T15:02:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 13 M 2024-04-14T15:02:51.669 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:52.008 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 1/2 2024-04-14T15:02:52.041 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 8.3 MB/s | 3.1 MB 00:00 2024-04-14T15:02:52.041 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:52.042 INFO:teuthology.orchestra.run.smithi045.stdout:Total 8.3 MB/s | 3.1 MB 00:00 2024-04-14T15:02:52.042 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:52.051 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:52.051 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:52.133 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:52.134 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:52.392 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-14T15:02:52.392 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:52.392 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:52.392 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-2778.ga971fd5f.el9.noarch 2024-04-14T15:02:52.392 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-14T15:02:52.393 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:52.393 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:52.577 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:52.625 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T15:02:52.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:52.704 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados-devel 2024-04-14T15:02:53.313 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:20 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.603 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:53.604 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:53.605 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:53.606 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:53.607 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:53.608 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:53.609 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.610 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.611 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.612 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:53.671 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:53.671 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:53.671 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 127 k 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:02:53.672 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:53.673 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 127 k 2024-04-14T15:02:53.673 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 456 k 2024-04-14T15:02:53.673 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:53.796 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:53.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:53.796 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:53.796 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:53.797 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:53.797 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:54.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2024-04-14T15:02:54.213 INFO:teuthology.orchestra.run.smithi071.stdout:librados-devel-19.0.0-2778.ga971fd5f.el9.x86_64 234 kB/s | 127 kB 00:00 2024-04-14T15:02:54.214 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:54.214 INFO:teuthology.orchestra.run.smithi071.stdout:Total 234 kB/s | 127 kB 00:00 2024-04-14T15:02:54.214 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:54.220 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:54.220 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:54.280 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:54.281 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:54.428 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:54.537 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:54.673 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:30 ago on Sun 14 Apr 2024 03:00:24 PM UTC. 2024-04-14T15:02:54.976 INFO:teuthology.orchestra.run.smithi045.stderr:Modular dependency problems: 2024-04-14T15:02:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:02:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 1: conflicting requests 2024-04-14T15:02:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.976 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 2: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 3: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 4: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 5: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 6: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 7: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 8: conflicting requests 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.977 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 9: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 10: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 11: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 12: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 13: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 14: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 15: conflicting requests 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.978 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 16: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 17: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 18: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 19: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 20: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 21: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 22: conflicting requests 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.979 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 23: conflicting requests 2024-04-14T15:02:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 24: conflicting requests 2024-04-14T15:02:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 25: conflicting requests 2024-04-14T15:02:54.980 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.981 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 26: conflicting requests 2024-04-14T15:02:54.981 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.981 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 27: conflicting requests 2024-04-14T15:02:54.981 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 28: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 29: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 30: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 31: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 32: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 33: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 34: conflicting requests 2024-04-14T15:02:54.982 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 35: conflicting requests 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 36: conflicting requests 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 37: conflicting requests 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 39: conflicting requests 2024-04-14T15:02:54.983 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.984 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.985 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:54.986 INFO:teuthology.orchestra.run.smithi045.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:55.004 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:55.049 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-14T15:02:55.050 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:55.050 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-04-14T15:02:55.050 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:55.050 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-14T15:02:55.050 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 170 k 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 170 k 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 494 k 2024-04-14T15:02:55.051 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:55.325 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:55.404 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 483 kB/s | 170 kB 00:00 2024-04-14T15:02:55.405 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:55.405 INFO:teuthology.orchestra.run.smithi045.stdout:Total 481 kB/s | 170 kB 00:00 2024-04-14T15:02:55.405 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-14T15:02:55.413 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-14T15:02:55.413 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-14T15:02:55.477 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-14T15:02:55.477 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-14T15:02:55.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs2 2024-04-14T15:02:55.630 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-14T15:02:55.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:56.162 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:56.178 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:23 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:02:56.472 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-14T15:02:56.474 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:56.474 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:56.474 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:56.474 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.475 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.476 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.477 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.478 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.479 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.480 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.481 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.482 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.483 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.483 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:56.483 INFO:teuthology.orchestra.run.smithi071.stdout:Package libcephfs2-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:56.544 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:56.545 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:02:56.545 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:56.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install libcephfs-devel 2024-04-14T15:02:56.723 DEBUG:teuthology.parallel:result is None 2024-04-14T15:02:57.245 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:24 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:57.545 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.546 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.547 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:57.548 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.549 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.550 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.551 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:57.552 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.553 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.554 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:57.617 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 32 k 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 32 k 2024-04-14T15:02:57.618 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 142 k 2024-04-14T15:02:57.619 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:02:57.885 INFO:teuthology.orchestra.run.smithi071.stdout:libcephfs-devel-19.0.0-2778.ga971fd5f.el9.x86_6 119 kB/s | 32 kB 00:00 2024-04-14T15:02:57.886 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:02:57.887 INFO:teuthology.orchestra.run.smithi071.stdout:Total 118 kB/s | 32 kB 00:00 2024-04-14T15:02:57.887 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:02:57.890 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:02:57.890 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:02:57.920 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:02:57.920 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:02:58.014 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:02:58.142 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:58.485 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:02:58.774 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:58.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librados2 2024-04-14T15:02:59.532 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:26 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.829 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.830 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:02:59.831 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:02:59.832 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.833 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.834 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:02:59.835 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:02:59.836 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.837 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:02:59.838 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:02:59.898 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:02:59.899 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:02:59.899 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:02:59.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install librbd1 2024-04-14T15:03:00.573 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:27 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:00.863 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:00.863 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:00.863 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:00.863 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:00.864 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:00.865 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.866 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:00.867 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.868 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.869 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:00.871 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:03:00.933 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:00.935 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:03:00.935 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:01.012 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rados 2024-04-14T15:03:01.607 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:28 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.894 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.895 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:01.896 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.897 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:01.898 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.899 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.900 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.901 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:01.902 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rados-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:03:01.962 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:01.964 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:03:01.964 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:02.042 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rgw 2024-04-14T15:03:02.641 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:29 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.934 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:02.935 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:02.936 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.937 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:02.938 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.939 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.940 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.941 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.942 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:02.943 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rgw-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:03:03.003 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:03.005 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:03:03.005 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:03.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-cephfs 2024-04-14T15:03:03.667 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:30 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.959 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.960 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.961 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.962 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.963 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:03.964 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.965 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.966 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.967 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:03.968 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-cephfs-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:03:04.027 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:04.028 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:03:04.028 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:04.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install python3-rbd 2024-04-14T15:03:04.704 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:31 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:04.995 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:04.996 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.997 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.998 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:04.999 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:05.000 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.001 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.002 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.003 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.004 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:05.005 INFO:teuthology.orchestra.run.smithi071.stdout:Package python3-rbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 is already installed. 2024-04-14T15:03:05.065 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:05.066 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-04-14T15:03:05.066 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:05.147 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-fuse 2024-04-14T15:03:05.741 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:32 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:06.027 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.028 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.029 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.030 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:06.031 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:06.032 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.033 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.034 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.035 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:06.036 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.036 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.036 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.036 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:06.096 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 85 k 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 85 k 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 231 k 2024-04-14T15:03:06.097 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:03:06.471 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-fuse-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 229 kB/s | 85 kB 00:00 2024-04-14T15:03:06.471 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:03:06.472 INFO:teuthology.orchestra.run.smithi071.stdout:Total 228 kB/s | 85 kB 00:00 2024-04-14T15:03:06.472 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:03:06.479 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:03:06.479 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:03:06.539 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:03:06.539 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:03:06.729 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:03:06.826 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:07.220 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:07.549 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:07.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-mirror 2024-04-14T15:03:08.416 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:35 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:08.712 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:08.713 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.714 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.715 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.716 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:08.717 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.718 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.719 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:08.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.720 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:08.781 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:08.782 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repo Size 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 3.1 M 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 3.1 M 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 13 M 2024-04-14T15:03:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:03:09.815 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-mirror-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 3.0 MB/s | 3.1 MB 00:01 2024-04-14T15:03:09.815 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:03:09.816 INFO:teuthology.orchestra.run.smithi071.stdout:Total 3.0 MB/s | 3.1 MB 00:01 2024-04-14T15:03:09.816 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:03:09.825 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:03:09.825 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:03:09.902 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:03:09.902 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:03:10.332 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:03:10.398 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-14T15:03:10.424 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:11.625 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:11.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install rbd-nbd 2024-04-14T15:03:12.484 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:02:39 ago on Sun 14 Apr 2024 03:00:33 PM UTC. 2024-04-14T15:03:12.776 INFO:teuthology.orchestra.run.smithi071.stderr:Modular dependency problems: 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 1: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240321091303:20125149.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 2: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 3: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 4: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 5: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240401160413:b754926a.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 6: conflicting requests 2024-04-14T15:03:12.777 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 7: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240307184541:49cc9d1b.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 8: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 9: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 10: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 11: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 12: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 13: conflicting requests 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.778 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 14: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 15: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 16: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 17: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 18: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 19: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 20: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 21: conflicting requests 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.779 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 22: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 23: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 24: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 25: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 26: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 27: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 28: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 29: conflicting requests 2024-04-14T15:03:12.780 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 30: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 31: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 32: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240319173245:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 33: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 34: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 35: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 36: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240314161907:9edba152.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 37: conflicting requests 2024-04-14T15:03:12.781 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 38: cannot install the best candidate for the job 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 39: conflicting requests 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.782 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 40: cannot install the best candidate for the job 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 41: cannot install the best candidate for the job 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 42: cannot install the best candidate for the job 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.783 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 43: cannot install the best candidate for the job 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 44: cannot install the best candidate for the job 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: Problem 45: cannot install the best candidate for the job 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.784 INFO:teuthology.orchestra.run.smithi071.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-04-14T15:03:12.845 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-14T15:03:12.845 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:12.845 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-04-14T15:03:12.845 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-2778.ga971fd5f.el9 ceph 170 k 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:Install 1 Package 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:12.846 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 170 k 2024-04-14T15:03:12.847 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 494 k 2024-04-14T15:03:12.847 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-14T15:03:13.067 INFO:teuthology.orchestra.run.smithi071.stdout:rbd-nbd-19.0.0-2778.ga971fd5f.el9.x86_64.rpm 774 kB/s | 170 kB 00:00 2024-04-14T15:03:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:03:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:Total 769 kB/s | 170 kB 00:00 2024-04-14T15:03:13.068 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-14T15:03:13.075 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-14T15:03:13.076 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-14T15:03:13.136 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-14T15:03:13.137 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-14T15:03:13.282 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-14T15:03:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:13.782 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:14.096 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 1/1 2024-04-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-2778.ga971fd5f.el9.x86_64 2024-04-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:03:14.097 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-14T15:03:14.303 DEBUG:teuthology.parallel:result is None 2024-04-14T15:03:14.303 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T15:03:14.451 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-14T15:03:14.475 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-2778.ga971fd5f.el9 2024-04-14T15:03:14.475 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2778.ga971fd5f.el9 2024-04-14T15:03:14.476 INFO:teuthology.task.install:The correct ceph version 19.0.0-2778.ga971fd5f is installed. 2024-04-14T15:03:14.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T15:03:14.616 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-14T15:03:14.636 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-2778.ga971fd5f.el9 2024-04-14T15:03:14.637 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2778.ga971fd5f.el9 2024-04-14T15:03:14.637 INFO:teuthology.task.install:The correct ceph version 19.0.0-2778.ga971fd5f is installed. 2024-04-14T15:03:14.638 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-14T15:03:14.638 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:03:14.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-14T15:03:14.671 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:03:14.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-14T15:03:14.703 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-14T15:03:14.704 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:03:14.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-14T15:03:14.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-14T15:03:14.812 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:03:14.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-14T15:03:14.838 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-14T15:03:14.906 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-14T15:03:14.906 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:03:14.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-14T15:03:14.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-14T15:03:15.004 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:03:15.004 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-14T15:03:15.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-14T15:03:15.097 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-14T15:03:15.098 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:03:15.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-14T15:03:15.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-14T15:03:15.196 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:03:15.196 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-14T15:03:15.223 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-14T15:03:15.289 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-14T15:03:15.385 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': 'a971fd5f5719038c70b817bafacd6d5400b1d888', 'cephadm_mode': 'cephadm-package'} 2024-04-14T15:03:15.386 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T15:03:15.386 INFO:tasks.cephadm:Cluster fsid is 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:03:15.386 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-14T15:03:15.386 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.45:6789]', 'mon.c': '[v1:172.21.15.45:6790]', 'mon.b': '[v1:172.21.15.71:6789]'} 2024-04-14T15:03:15.386 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-04-14T15:03:15.387 INFO:tasks.cephadm:First mgr is y 2024-04-14T15:03:15.387 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-14T15:03:15.387 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-04-14T15:03:15.414 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-04-14T15:03:15.451 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 on all hosts... 2024-04-14T15:03:15.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 pull 2024-04-14T15:03:15.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 pull 2024-04-14T15:03:16.289 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888... 2024-04-14T15:03:16.386 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888... 2024-04-14T15:04:04.352 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-14T15:04:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev)", 2024-04-14T15:04:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53", 2024-04-14T15:04:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-04-14T15:04:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608" 2024-04-14T15:04:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-04-14T15:04:04.354 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-14T15:04:31.038 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-14T15:04:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev)", 2024-04-14T15:04:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53", 2024-04-14T15:04:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-04-14T15:04:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608" 2024-04-14T15:04:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-14T15:04:31.039 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-14T15:04:31.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-04-14T15:04:31.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-04-14T15:04:31.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-04-14T15:04:31.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-04-14T15:04:31.206 INFO:tasks.cephadm:Writing seed config... 2024-04-14T15:04:31.207 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-14T15:04:31.207 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [global] ms type = async 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-04-14T15:04:31.208 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-14T15:04:31.209 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:04:31.209 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-14T15:04:31.224 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1eba33fc-fa70-11ee-bc8e-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-04-14T15:04:31.225 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service 2024-04-14T15:04:31.267 DEBUG:teuthology.orchestra.run.smithi045:mgr.y> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y.service 2024-04-14T15:04:31.309 INFO:tasks.cephadm:Bootstrapping... 2024-04-14T15:04:31.310 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 -v bootstrap --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.45:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:04:31.506 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-14T15:04:31.506 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888', '-v', 'bootstrap', '--fsid', '1eba33fc-fa70-11ee-bc8e-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.45:6789]', '--skip-admin-label'] 2024-04-14T15:04:31.507 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-14T15:04:31.507 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-04-14T15:04:31.531 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-14T15:04:31.531 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-04-14T15:04:31.531 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-04-14T15:04:31.539 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-14T15:04:31.539 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-14T15:04:31.544 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-14T15:04:31.545 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:04:31.551 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-14T15:04:31.557 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-14T15:04:31.557 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-14T15:04:31.557 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-04-14T15:04:31.581 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-14T15:04:31.581 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-14T15:04:31.581 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-04-14T15:04:31.581 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-04-14T15:04:31.586 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-14T15:04:31.587 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-14T15:04:31.592 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-14T15:04:31.592 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:04:31.598 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-14T15:04:31.604 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-14T15:04:31.604 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-14T15:04:31.604 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-04-14T15:04:31.604 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:31.605 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140072401003184 on /run/cephadm/1eba33fc-fa70-11ee-bc8e-c7b262605968.lock 2024-04-14T15:04:31.605 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140072401003184 acquired on /run/cephadm/1eba33fc-fa70-11ee-bc8e-c7b262605968.lock 2024-04-14T15:04:31.605 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-04-14T15:04:31.606 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:6789], mon addrv is [v1:172.21.15.45:6789] 2024-04-14T15:04:31.609 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-04-14T15:04:31.609 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-04-14T15:04:31.611 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-14T15:04:31.611 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-14T15:04:31.612 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-14T15:04:31.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-14T15:04:31.615 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-14T15:04:31.615 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-04-14T15:04:31.615 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-14T15:04:31.616 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888... 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888... 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:4a9a0f674f5d00de2fb0d6279a6376d7e805966559c8ffba59b1bc2aa8fd87e7 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:77070339c81bcb5c53bbbf3f31a47f7187e5393cb0e096452852a31192d97570 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 2024-04-14T15:04:32.175 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-14T15:04:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev) 2024-04-14T15:04:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev) 2024-04-14T15:04:33.163 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-04-14T15:04:33.986 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-04-14T15:04:33.986 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-04-14T15:04:35.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCC8BtmbwSqEhAAM4n37cwm5ugaC6+sS98/cA== 2024-04-14T15:04:35.948 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCD8BtmEy/9ExAAy8BAM7ob32wae7r27MIHdg== 2024-04-14T15:04:36.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQCE8BtmDG5XDhAAwrDij9cngwz9x7X88n1f0g== 2024-04-14T15:04:36.833 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-04-14T15:04:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-14T15:04:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-14T15:04:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:04:37.678 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-04-14T15:04:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-04-14T15:04:38.863 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-14T15:04:39.034 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-14T15:04:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target → /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target. 2024-04-14T15:04:39.202 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target → /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target. 2024-04-14T15:04:39.422 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a 2024-04-14T15:04:39.422 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service: Unit ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service not loaded. 2024-04-14T15:04:39.579 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target.wants/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@.service. 2024-04-14T15:04:40.073 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:40 smithi045 ceph-mon[29245]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-14T15:04:40.139 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-14T15:04:40.139 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-14T15:04:40.147 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-14T15:04:40.147 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:04:40.147 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-14T15:04:40.147 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-14T15:04:40.147 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-04-14T15:04:40.148 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-04-14T15:04:40.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:40 smithi045 bash[29225]: ebd66808365cf1929f654f3d0d847f9fbd687f76884b78af86574519036dd8be 2024-04-14T15:04:40.377 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:40 smithi045 systemd[1]: Started Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: monmap epoch 1 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: last_changed 2024-04-14T15:04:37.102829+0000 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: min_mon_release 19 (squid) 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: election_strategy: 1 2024-04-14T15:04:41.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:04:41.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: fsmap 2024-04-14T15:04:41.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: osdmap e1: 0 total, 0 up, 0 in 2024-04-14T15:04:41.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: mgrmap e1: no daemons active 2024-04-14T15:04:41.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:41 smithi045 ceph-mon[29245]: from='client.? v1:172.21.15.45:0/3144050826' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.615255s) 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-14T15:04:41.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.600+0000 7fee8d669700 1 Processor -- start 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.600+0000 7fee8d669700 1 -- start start 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.600+0000 7fee8d669700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fee8807a700 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee86ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee88107340 0x7fee88109730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46882/0 (socket says 172.21.15.45:46882) 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee86ffd700 1 -- 172.21.15.45:0/1718984152 learned_addr learned my addr 172.21.15.45:0/1718984152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 863029860 0 0) 0x7fee8807a700 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee700035b0 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2174560128 0 0) 0x7fee700035b0 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8810be60 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee7c001580 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee7c001850 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.601+0000 7fee85ffb700 1 -- 172.21.15.45:0/1718984152 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee7c001bd0 con 0x7fee88107340 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.602+0000 7fee8d669700 1 -- 172.21.15.45:0/1718984152 >> v1:172.21.15.45:6789/0 conn(0x7fee88107340 legacy=0x7fee88109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:41.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.602+0000 7fee8d669700 1 -- 172.21.15.45:0/1718984152 shutdown_connections 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.602+0000 7fee8d669700 1 -- 172.21.15.45:0/1718984152 >> 172.21.15.45:0/1718984152 conn(0x7fee88100d50 msgr2=0x7fee88103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.602+0000 7fee8d669700 1 -- 172.21.15.45:0/1718984152 shutdown_connections 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.602+0000 7fee8d669700 1 -- 172.21.15.45:0/1718984152 wait complete. 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee8d669700 1 Processor -- start 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee8d669700 1 -- start start 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee8d669700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fee8819a3e0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee86ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee88107340 0x7fee88199cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46888/0 (socket says 172.21.15.45:46888) 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee86ffd700 1 -- 172.21.15.45:0/3144050826 learned_addr learned my addr 172.21.15.45:0/3144050826 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787387749 0 0) 0x7fee8819a3e0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.603+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee700031f0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1671428982 0 0) 0x7fee700031f0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee8819a3e0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee7c001540 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3143483631 0 0) 0x7fee8819a3e0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8819a5b0 con 0x7fee88107340 2024-04-14T15:04:41.244 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fee8819a8a0 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fee8819e3e0 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fee7c001850 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee7c004c80 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fee7c005270 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.604+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7fee7c006a80 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.605+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee74005150 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.607+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7fee7c005740 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.682+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "status"} v 0) -- 0x7fee74005c50 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.683+0000 7fee6ffff700 1 -- 172.21.15.45:0/3144050826 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (unknown 1155462804 0 55692663) 0x7fee7c005980 con 0x7fee88107340 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.684+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 >> v1:172.21.15.45:6789/0 conn(0x7fee88107340 legacy=0x7fee88199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.684+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 shutdown_connections 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.684+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 >> 172.21.15.45:0/3144050826 conn(0x7fee88100d50 msgr2=0x7fee88190da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.684+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 shutdown_connections 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:40.684+0000 7fee8d669700 1 -- 172.21.15.45:0/3144050826 wait complete. 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-04-14T15:04:41.245 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.45:6789] 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-14T15:04:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44aee40700 1 Processor -- start 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44aee40700 1 -- start start 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44aee40700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f44a810ad30 con 0x7f44a8106900 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44acbdc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f44a8106900 0x7f44a8106d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54530/0 (socket says 172.21.15.45:54530) 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44acbdc700 1 -- 172.21.15.45:0/3437276106 learned_addr learned my addr 172.21.15.45:0/3437276106 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:42.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.636+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308440082 0 0) 0x7f44a810ad30 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44940035b0 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634131470 0 0) 0x7f44940035b0 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a810bf10 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f44a0001900 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44a0001b60 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44a77fe700 1 -- 172.21.15.45:0/3437276106 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f44a0001ee0 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44aee40700 1 -- 172.21.15.45:0/3437276106 >> v1:172.21.15.45:6789/0 conn(0x7f44a8106900 legacy=0x7f44a8106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44aee40700 1 -- 172.21.15.45:0/3437276106 shutdown_connections 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44aee40700 1 -- 172.21.15.45:0/3437276106 >> 172.21.15.45:0/3437276106 conn(0x7f44a8101e80 msgr2=0x7f44a81042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44aee40700 1 -- 172.21.15.45:0/3437276106 shutdown_connections 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.637+0000 7f44aee40700 1 -- 172.21.15.45:0/3437276106 wait complete. 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.638+0000 7f44aee40700 1 Processor -- start 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.638+0000 7f44aee40700 1 -- start start 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.638+0000 7f44aee40700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f44a819a3d0 con 0x7f44a8106900 2024-04-14T15:04:42.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.638+0000 7f44acbdc700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f44a8106900 0x7f44a8199cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54532/0 (socket says 172.21.15.45:54532) 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.638+0000 7f44acbdc700 1 -- 172.21.15.45:0/42681481 learned_addr learned my addr 172.21.15.45:0/42681481 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671412900 0 0) 0x7f44a819a3d0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44940031f0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 954950741 0 0) 0x7f44940031f0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f44a819a3d0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f44a0001800 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1869370560 0 0) 0x7f44a819a3d0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a819a5a0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f44a819a890 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f44a0004ea0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f44a00051e0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f44a0005790 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f44a819e3e0 con 0x7f44a8106900 2024-04-14T15:04:42.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.639+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f44a0005ff0 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.640+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a810b020 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.642+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7f44a0001ad0 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.714+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f44a810b2a0 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.716+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+317 (unknown 1213389831 0 3810165280) 0x7f44a0006300 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.717+0000 7f44a5ffb700 1 -- 172.21.15.45:0/42681481 <== mon.0 v1:172.21.15.45:6789/0 11 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f44a0006560 con 0x7f44a8106900 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.718+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 >> v1:172.21.15.45:6789/0 conn(0x7f44a8106900 legacy=0x7f44a8199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.718+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 shutdown_connections 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.718+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 >> 172.21.15.45:0/42681481 conn(0x7f44a8101e80 msgr2=0x7f44a8190e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.718+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 shutdown_connections 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:41.718+0000 7f44aee40700 1 -- 172.21.15.45:0/42681481 wait complete. 2024-04-14T15:04:42.243 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-04-14T15:04:42.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:42 smithi045 ceph-mon[29245]: from='client.? v1:172.21.15.45:0/42681481' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-14T15:04:42.357 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:42 smithi045 ceph-mon[29245]: from='client.? v1:172.21.15.45:0/42681481' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-14T15:04:43.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:43 smithi045 ceph-mon[29245]: from='client.? v1:172.21.15.45:0/3682812408' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5be51e700 1 Processor -- start 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5be51e700 1 -- start start 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5be51e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5b810ad10 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5b7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5b81068e0 0x7fd5b8106d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54534/0 (socket says 172.21.15.45:54534) 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5b7fff700 1 -- 172.21.15.45:0/3707276498 learned_addr learned my addr 172.21.15.45:0/3707276498 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.643+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1737385944 0 0) 0x7fd5b810ad10 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5a00035b0 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2483956099 0 0) 0x7fd5a00035b0 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b810bef0 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd5a8001580 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fd5a8001d20 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5b6ffd700 1 -- 172.21.15.45:0/3707276498 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd5a8005960 con 0x7fd5b81068e0 2024-04-14T15:04:43.304 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5be51e700 1 -- 172.21.15.45:0/3707276498 >> v1:172.21.15.45:6789/0 conn(0x7fd5b81068e0 legacy=0x7fd5b8106d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5be51e700 1 -- 172.21.15.45:0/3707276498 shutdown_connections 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5be51e700 1 -- 172.21.15.45:0/3707276498 >> 172.21.15.45:0/3707276498 conn(0x7fd5b8101e60 msgr2=0x7fd5b81042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.644+0000 7fd5be51e700 1 -- 172.21.15.45:0/3707276498 shutdown_connections 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.645+0000 7fd5be51e700 1 -- 172.21.15.45:0/3707276498 wait complete. 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.645+0000 7fd5be51e700 1 Processor -- start 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.645+0000 7fd5be51e700 1 -- start start 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.645+0000 7fd5be51e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd5b819ac00 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd5b81068e0 0x7fd5b819a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54546/0 (socket says 172.21.15.45:54546) 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b7fff700 1 -- 172.21.15.45:0/3682812408 learned_addr learned my addr 172.21.15.45:0/3682812408 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158489979 0 0) 0x7fd5b819ac00 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5a0005530 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1088798095 0 0) 0x7fd5a0005530 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5b819ac00 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd5a8004870 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1866454618 0 0) 0x7fd5b819ac00 con 0x7fd5b81068e0 2024-04-14T15:04:43.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b819add0 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b819b0c0 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5b819ec10 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fd5a8001c00 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd5a8005f10 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fd5a80064c0 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.646+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7fd5a8007ba0 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.647+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a4005150 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.649+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7fd5a8004a40 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.720+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd5a4005440 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.720+0000 7fd5b57fa700 1 -- 172.21.15.45:0/3682812408 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+147 (unknown 1452402520 0 1142551679) 0x7fd5a8004d00 con 0x7fd5b81068e0 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.721+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 >> v1:172.21.15.45:6789/0 conn(0x7fd5b81068e0 legacy=0x7fd5b819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.721+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 shutdown_connections 2024-04-14T15:04:43.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.721+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 >> 172.21.15.45:0/3682812408 conn(0x7fd5b8101e60 msgr2=0x7fd5b81042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:43.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.721+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 shutdown_connections 2024-04-14T15:04:43.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:42.721+0000 7fd5be51e700 1 -- 172.21.15.45:0/3682812408 wait complete. 2024-04-14T15:04:43.307 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-04-14T15:04:43.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:43 smithi045 systemd[1]: Stopping Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:04:43.983 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a[29241]: 2024-04-14T15:04:43.566+0000 7f07a38c1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:04:43.983 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a[29241]: 2024-04-14T15:04:43.566+0000 7f07a38c1700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-14T15:04:43.983 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:43 smithi045 podman[29500]: 2024-04-14 15:04:43.771066649 +0000 UTC m=+0.302775065 container died ebd66808365cf1929f654f3d0d847f9fbd687f76884b78af86574519036dd8be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-14T15:04:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29500]: 2024-04-14 15:04:44.057786485 +0000 UTC m=+0.589494883 container cleanup ebd66808365cf1929f654f3d0d847f9fbd687f76884b78af86574519036dd8be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-14T15:04:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 bash[29500]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a 2024-04-14T15:04:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29514]: 2024-04-14 15:04:44.200597529 +0000 UTC m=+0.427550220 container remove ebd66808365cf1929f654f3d0d847f9fbd687f76884b78af86574519036dd8be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-14T15:04:44.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-14T15:04:44.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 systemd[1]: Stopped Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:04:44.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 systemd[1]: Starting Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:04:44.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29605]: 2024-04-14 15:04:44.618040516 +0000 UTC m=+0.110910164 container create 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-a971fd5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2) 2024-04-14T15:04:44.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29605]: 2024-04-14 15:04:44.524946768 +0000 UTC m=+0.017816432 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 2024-04-14T15:04:44.924 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29605]: 2024-04-14 15:04:44.83791243 +0000 UTC m=+0.330782089 container init 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 podman[29605]: 2024-04-14 15:04:44.840934859 +0000 UTC m=+0.333804510 container start 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: set uid:gid to 167:167 (ceph:ceph) 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev), process ceph-mon, pid 2 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: pidfile_write: ignore empty --pid-file 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: load: jerasure load: lrc 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: RocksDB version: 7.9.2 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Git sha 0 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Compile date 2024-04-06 20:28:24 2024-04-14T15:04:44.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: DB SUMMARY 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: DB Session ID: FQDH2S2143M0FITYJISZ 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: CURRENT file: CURRENT 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: IDENTITY file: IDENTITY 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 89169 ; 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.error_if_exists: 0 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.create_if_missing: 0 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.paranoid_checks: 1 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.env: 0x557f52422c60 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.fs: PosixFileSystem 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.info_log: 0x557f540bbc40 2024-04-14T15:04:44.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_file_opening_threads: 16 2024-04-14T15:04:44.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.statistics: (nil) 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.use_fsync: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_log_file_size: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.keep_log_file_num: 1000 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.recycle_log_file_num: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_fallocate: 1 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_mmap_reads: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_mmap_writes: 0 2024-04-14T15:04:45.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.use_direct_reads: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.create_missing_column_families: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.db_log_dir: 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.wal_dir: 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.advise_random_on_open: 1 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.db_write_buffer_size: 0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.write_buffer_manager: 0x557f5408c1e0 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-14T15:04:45.001 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.rate_limiter: (nil) 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.wal_recovery_mode: 2 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enable_thread_tracking: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enable_pipelined_write: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.unordered_write: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.row_cache: None 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.wal_filter: None 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-14T15:04:45.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_ingest_behind: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.two_write_queues: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.manual_wal_flush: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.wal_compression: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.atomic_flush: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.log_readahead_size: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.best_efforts_recovery: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.allow_data_in_errors: 0 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.db_host_id: __hostname__ 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_background_jobs: 2 2024-04-14T15:04:45.003 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_background_compactions: -1 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_subcompactions: 1 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_total_wal_size: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_open_files: -1 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bytes_per_sync: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_readahead_size: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_background_flushes: -1 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Compression algorithms supported: 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-14T15:04:45.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kZSTD supported: 0 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kXpressCompression supported: 0 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kLZ4HCCompression supported: 1 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kZlibCompression supported: 1 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kSnappyCompression supported: 1 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kLZ4Compression supported: 1 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: kBZip2Compression supported: 0 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.merge_operator: 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_filter: None 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_filter_factory: None 2024-04-14T15:04:45.005 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.sst_partitioner_factory: None 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557f540bbe00) 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x557f540bf090 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-04-14T15:04:45.006 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-04-14T15:04:45.007 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.write_buffer_size: 33554432 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_write_buffer_number: 2 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression: NoCompression 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression: Disabled 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.prefix_extractor: nullptr 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.num_levels: 7 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-14T15:04:45.008 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.level: 32767 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.strategy: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:04:45.009 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.enabled: false 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.target_file_size_base: 67108864 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-14T15:04:45.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.arena_block_size: 1048576 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.disable_auto_compactions: 0 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-14T15:04:45.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.inplace_update_support: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.bloom_locality: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.max_successive_merges: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.paranoid_file_checks: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.force_consistency_checks: 1 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.report_bg_io_stats: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.ttl: 2592000 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-14T15:04:45.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enable_blob_files: false 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.min_blob_size: 0 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_file_size: 268435456 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.blob_file_starting_level: 0 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 66042715-42f8-4d05-8133-b4cbcd3dd722 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107084870031, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107084870901, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 270, "table_properties": {"data_size": 83631, "index_size": 231, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10877, "raw_average_key_size": 47, "raw_value_size": 77530, "raw_average_value_size": 338, "num_data_blocks": 10, "num_entries": 229, "num_filter_entries": 229, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713107084, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "66042715-42f8-4d05-8133-b4cbcd3dd722", "db_session_id": "FQDH2S2143M0FITYJISZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-14T15:04:45.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107084870988, "job": 1, "event": "recovery_finished"} 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557f541ae000 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: DB pointer 0x557f54198000 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:04:45.014 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 85.30 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 85.30 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.015 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 27.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 27.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x557f540bf090#2 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: starting mon.a rank 0 at public addrs v1:172.21.15.45:6789/0 at bind addrs v1:172.21.15.45:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???) e1 preinit fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).mds e1 new map 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).mds e1 print_map 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-04-14T15:04:45.016 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 bash[29605]: 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 systemd[1]: Started Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: monmap epoch 1 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: last_changed 2024-04-14T15:04:37.102829+0000 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:04:45.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: min_mon_release 19 (squid) 2024-04-14T15:04:45.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: election_strategy: 1 2024-04-14T15:04:45.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:04:45.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: fsmap 2024-04-14T15:04:45.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: osdmap e1: 0 total, 0 up, 0 in 2024-04-14T15:04:45.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:44 smithi045 ceph-mon[29624]: mgrmap e1: no daemons active 2024-04-14T15:04:46.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.385+0000 7fc377ef6700 1 Processor -- start 2024-04-14T15:04:46.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.385+0000 7fc377ef6700 1 -- start start 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc377ef6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc37007a6e0 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc375c92700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc370105170 0x7fc370105590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54550/0 (socket says 172.21.15.45:54550) 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc375c92700 1 -- 172.21.15.45:0/3821251873 learned_addr learned my addr 172.21.15.45:0/3821251873 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2199448803 0 0) 0x7fc37007a6e0 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc35c0035b0 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4035828943 0 0) 0x7fc35c0035b0 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.386+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc370106cb0 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc36c001580 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fc36c001d20 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc374c90700 1 -- 172.21.15.45:0/3821251873 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc36c005960 con 0x7fc370105170 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc377ef6700 1 -- 172.21.15.45:0/3821251873 >> v1:172.21.15.45:6789/0 conn(0x7fc370105170 legacy=0x7fc370105590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc377ef6700 1 -- 172.21.15.45:0/3821251873 shutdown_connections 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc377ef6700 1 -- 172.21.15.45:0/3821251873 >> 172.21.15.45:0/3821251873 conn(0x7fc370100d30 msgr2=0x7fc3701031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc377ef6700 1 -- 172.21.15.45:0/3821251873 shutdown_connections 2024-04-14T15:04:46.017 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.387+0000 7fc377ef6700 1 -- 172.21.15.45:0/3821251873 wait complete. 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc377ef6700 1 Processor -- start 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc377ef6700 1 -- start start 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc377ef6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc37019cd80 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc375c92700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc370105170 0x7fc37019c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54552/0 (socket says 172.21.15.45:54552) 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc375c92700 1 -- 172.21.15.45:0/1426182347 learned_addr learned my addr 172.21.15.45:0/1426182347 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2626903917 0 0) 0x7fc37019cd80 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.388+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc35c005530 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 972735817 0 0) 0x7fc35c005530 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc37019cd80 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc36c004870 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 834986996 0 0) 0x7fc37019cd80 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37019cf50 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc37019d240 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc3701a0d90 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fc36c001190 con 0x7fc370105170 2024-04-14T15:04:46.018 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc36c005f60 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fc36c0065b0 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.389+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7fc36c007cf0 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.390+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc354005150 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.392+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7fc36c001c00 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.461+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fc354005440 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.463+0000 7fc366ffd700 1 -- 172.21.15.45:0/1426182347 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (unknown 534127980 0 0) 0x7fc36c004ab0 con 0x7fc370105170 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.464+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 >> v1:172.21.15.45:6789/0 conn(0x7fc370105170 legacy=0x7fc37019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.464+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 shutdown_connections 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.464+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 >> 172.21.15.45:0/1426182347 conn(0x7fc370100d30 msgr2=0x7fc3701031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.464+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 shutdown_connections 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:45.464+0000 7fc377ef6700 1 -- 172.21.15.45:0/1426182347 wait complete. 2024-04-14T15:04:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-14T15:04:46.044 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:04:46.044 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-04-14T15:04:46.044 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-14T15:04:46.045 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-14T15:04:46.295 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y 2024-04-14T15:04:46.295 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y.service: Unit ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y.service not loaded. 2024-04-14T15:04:46.442 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968.target.wants/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@.service. 2024-04-14T15:04:46.663 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1426182347' entity='client.admin' 2024-04-14T15:04:46.957 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:46 smithi045 podman[29843]: 2024-04-14 15:04:46.876372739 +0000 UTC m=+0.317408586 container start 2f17f04e2184d5ae197fb0cbad8c5741ffe248117f81211ba2577b29f1cf0358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:04:46.972 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-14T15:04:46.972 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-14T15:04:46.980 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-14T15:04:46.980 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:04:46.980 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-14T15:04:46.980 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-14T15:04:46.989 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-14T15:04:46.989 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-04-14T15:04:46.997 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-04-14T15:04:47.210 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:46 smithi045 bash[29843]: 2f17f04e2184d5ae197fb0cbad8c5741ffe248117f81211ba2577b29f1cf0358 2024-04-14T15:04:47.210 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:46 smithi045 systemd[1]: Started Ceph mgr.y for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:04:47.548 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/318392079' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-14T15:04:47.548 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:47 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:47.448+0000 7f3bac55e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:04:47.549 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:47 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:47.548+0000 7f3bac55e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "1eba33fc-fa70-11ee-bc8e-c7b262605968", 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-14T15:04:48.061 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-14T15:04:48.068 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-14T15:04:48.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-14T15:04:40.069723+0000", 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.386+0000 7f4d25c23700 1 Processor -- start 2024-04-14T15:04:48.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.386+0000 7f4d25c23700 1 -- start start 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.386+0000 7f4d25c23700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4d2007a6e0 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.386+0000 7f4d1f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d20107380 0x7f4d20109770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54588/0 (socket says 172.21.15.45:54588) 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.386+0000 7f4d1f7fe700 1 -- 172.21.15.45:0/2870378742 learned_addr learned my addr 172.21.15.45:0/2870378742 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066231737 0 0) 0x7f4d2007a6e0 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d080035b0 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4290979419 0 0) 0x7f4d080035b0 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2010bea0 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f4d10001580 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f4d10001d20 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.387+0000 7f4d1e7fc700 1 -- 172.21.15.45:0/2870378742 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f4d10005960 con 0x7f4d20107380 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 -- 172.21.15.45:0/2870378742 >> v1:172.21.15.45:6789/0 conn(0x7f4d20107380 legacy=0x7f4d20109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 -- 172.21.15.45:0/2870378742 shutdown_connections 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 -- 172.21.15.45:0/2870378742 >> 172.21.15.45:0/2870378742 conn(0x7f4d20100d30 msgr2=0x7f4d201031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 -- 172.21.15.45:0/2870378742 shutdown_connections 2024-04-14T15:04:48.071 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 -- 172.21.15.45:0/2870378742 wait complete. 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.388+0000 7f4d25c23700 1 Processor -- start 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d25c23700 1 -- start start 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d25c23700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4d201a04e0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4d20107380 0x7f4d20106c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54600/0 (socket says 172.21.15.45:54600) 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1f7fe700 1 -- 172.21.15.45:0/318392079 learned_addr learned my addr 172.21.15.45:0/318392079 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4044784309 0 0) 0x7f4d201a04e0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d08005530 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1405380040 0 0) 0x7f4d08005530 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d201a04e0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f4d10004870 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1810746369 0 0) 0x7f4d201a04e0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d201a16c0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.389+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4d201a0710 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.390+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4d201a0c50 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.390+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f4d10004ab0 con 0x7f4d20107380 2024-04-14T15:04:48.072 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.390+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f4d10006250 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.390+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f4d10006860 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.390+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f4d10007f70 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.391+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d20064110 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.393+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7f4d100056d0 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.467+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d2002e0c0 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.467+0000 7f4d1cff9700 1 -- 172.21.15.45:0/318392079 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 2042533091) 0x7f4d10005950 con 0x7f4d20107380 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.469+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 >> v1:172.21.15.45:6789/0 conn(0x7f4d20107380 legacy=0x7f4d20106c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.469+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 shutdown_connections 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.469+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 >> 172.21.15.45:0/318392079 conn(0x7f4d20100d30 msgr2=0x7f4d20191200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.469+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 shutdown_connections 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:47.469+0000 7f4d25c23700 1 -- 172.21.15.45:0/318392079 wait complete. 2024-04-14T15:04:48.073 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-04-14T15:04:48.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.220+0000 7f3bac55e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:04:48.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.314+0000 7f3bac55e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:04:48.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.458+0000 7f3bac55e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:04:48.914 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.558+0000 7f3bac55e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:04:48.914 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.643+0000 7f3bac55e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:04:48.914 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:48 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:48.913+0000 7f3bac55e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:04:49.218 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:49.027+0000 7f3bac55e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:04:49.218 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:49.114+0000 7f3bac55e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:04:49.218 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:49.217+0000 7f3bac55e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:04:49.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:49.404+0000 7f3bac55e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:04:50.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:50.406+0000 7f3bac55e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:04:50.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3059920269' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-14T15:04:50.841 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:50.544+0000 7f3bac55e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:04:51.184 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:50.891+0000 7f3bac55e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:04:51.185 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:50.983+0000 7f3bac55e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:04:51.185 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:51 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:51.184+0000 7f3bac55e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:04:51.240 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "1eba33fc-fa70-11ee-bc8e-c7b262605968", 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.241 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:51.242 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-14T15:04:51.243 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-14T15:04:51.252 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-14T15:04:51.252 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-14T15:04:51.253 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-14T15:04:51.254 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-14T15:04:40.069723+0000", 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f9468e57700 1 Processor -- start 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f9468e57700 1 -- start start 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f9468e57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f946407a6a0 con 0x7f94641051b0 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f946259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94641051b0 0x7f94641055d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54610/0 (socket says 172.21.15.45:54610) 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f946259c700 1 -- 172.21.15.45:0/3600412394 learned_addr learned my addr 172.21.15.45:0/3600412394 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3849029174 0 0) 0x7f946407a6a0 con 0x7f94641051b0 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.529+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f944c0035b0 con 0x7f94641051b0 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3614746017 0 0) 0x7f944c0035b0 con 0x7f94641051b0 2024-04-14T15:04:51.256 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9464106cf0 con 0x7f94641051b0 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9454001580 con 0x7f94641051b0 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f9454001c50 con 0x7f94641051b0 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f946159a700 1 -- 172.21.15.45:0/3600412394 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9454005960 con 0x7f94641051b0 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.530+0000 7f9468e57700 1 -- 172.21.15.45:0/3600412394 >> v1:172.21.15.45:6789/0 conn(0x7f94641051b0 legacy=0x7f94641055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 -- 172.21.15.45:0/3600412394 shutdown_connections 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 -- 172.21.15.45:0/3600412394 >> 172.21.15.45:0/3600412394 conn(0x7f9464100d30 msgr2=0x7f94641031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 -- 172.21.15.45:0/3600412394 shutdown_connections 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 -- 172.21.15.45:0/3600412394 wait complete. 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 Processor -- start 2024-04-14T15:04:51.257 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.531+0000 7f9468e57700 1 -- start start 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f9468e57700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f946419ce00 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f946259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f94641051b0 0x7f946419c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54622/0 (socket says 172.21.15.45:54622) 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f946259c700 1 -- 172.21.15.45:0/3059920269 learned_addr learned my addr 172.21.15.45:0/3059920269 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1756242737 0 0) 0x7f946419ce00 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f944c005580 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1643991698 0 0) 0x7f944c005580 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f946419ce00 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9454001080 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 227649691 0 0) 0x7f946419ce00 con 0x7f94641051b0 2024-04-14T15:04:51.258 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946419cfd0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.532+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f946419d2c0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.533+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f94641a0e10 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.533+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f9454001bb0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.533+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9454005fb0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.533+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f94540065c0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.533+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f9454007cd0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.534+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9444005150 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.536+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7f94540055f0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.610+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9444005440 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.610+0000 7f94537fe700 1 -- 172.21.15.45:0/3059920269 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 373186141) 0x7f94540058b0 con 0x7f94641051b0 2024-04-14T15:04:51.259 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.611+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 >> v1:172.21.15.45:6789/0 conn(0x7f94641051b0 legacy=0x7f946419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:51.260 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.611+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 shutdown_connections 2024-04-14T15:04:51.260 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.611+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 >> 172.21.15.45:0/3059920269 conn(0x7f9464100d30 msgr2=0x7f94641031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:51.260 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.611+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 shutdown_connections 2024-04-14T15:04:51.260 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:50.611+0000 7f9468e57700 1 -- 172.21.15.45:0/3059920269 wait complete. 2024-04-14T15:04:51.260 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-04-14T15:04:51.454 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:51 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:51.374+0000 7f3bac55e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:04:51.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:51 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:51.677+0000 7f3bac55e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:04:52.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:51 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:51.938+0000 7f3bac55e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:04:52.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:52.178+0000 7f3bac55e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:04:53.366 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:53.006+0000 7f3bac55e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:04:53.645 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:53.594+0000 7f3bac55e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:04:53.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2253008754' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-14T15:04:53.927 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:53.698+0000 7f3bac55e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:04:53.927 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:04:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:04:53.925+0000 7f3bac55e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "1eba33fc-fa70-11ee-bc8e-c7b262605968", 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-14T15:04:54.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-14T15:04:54.198 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-14T15:04:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:04:54.207 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-14T15:04:40.069723+0000", 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.643+0000 7f45ccf8b700 1 Processor -- start 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.643+0000 7f45ccf8b700 1 -- start start 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.644+0000 7f45ccf8b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f45c8109c20 con 0x7f45c8106d80 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.644+0000 7f45c659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45c8106d80 0x7f45c81071a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57418/0 (socket says 172.21.15.45:57418) 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.644+0000 7f45c659c700 1 -- 172.21.15.45:0/2171782489 learned_addr learned my addr 172.21.15.45:0/2171782489 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.644+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437483059 0 0) 0x7f45c8109c20 con 0x7f45c8106d80 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.644+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b00035b0 con 0x7f45c8106d80 2024-04-14T15:04:54.208 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 831983149 0 0) 0x7f45b00035b0 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c810ae00 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f45b8001580 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f45b8001c50 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45c559a700 1 -- 172.21.15.45:0/2171782489 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f45b8005960 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.645+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2171782489 >> v1:172.21.15.45:6789/0 conn(0x7f45c8106d80 legacy=0x7f45c81071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2171782489 shutdown_connections 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2171782489 >> 172.21.15.45:0/2171782489 conn(0x7f45c8100550 msgr2=0x7f45c8102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2171782489 shutdown_connections 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2171782489 wait complete. 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 Processor -- start 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- start start 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.646+0000 7f45ccf8b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f45c8105370 con 0x7f45c8106d80 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45c659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45c8106d80 0x7f45c819d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57430/0 (socket says 172.21.15.45:57430) 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45c659c700 1 -- 172.21.15.45:0/2253008754 learned_addr learned my addr 172.21.15.45:0/2253008754 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:04:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291648246 0 0) 0x7f45c8105370 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45b0005580 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1539765792 0 0) 0x7f45b0005580 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f45c8105370 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f45b8001080 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 254276812 0 0) 0x7f45c8105370 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c8105540 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f45c819e980 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f45c819ed20 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f45b8001b90 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.647+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f45b8005fb0 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.648+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f45b80065a0 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.648+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f45b8007d30 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.648+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a8005150 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.652+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (unknown 1092875540 0 2268998900) 0x7f45b80055f0 con 0x7f45c8106d80 2024-04-14T15:04:54.210 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.726+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f45a8005440 con 0x7f45c8106d80 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.726+0000 7f45b77fe700 1 -- 172.21.15.45:0/2253008754 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 2565916060) 0x7f45b8005870 con 0x7f45c8106d80 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.727+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 >> v1:172.21.15.45:6789/0 conn(0x7f45c8106d80 legacy=0x7f45c819d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.727+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 shutdown_connections 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.727+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 >> 172.21.15.45:0/2253008754 conn(0x7f45c8100550 msgr2=0x7f45c8190070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.727+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 shutdown_connections 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:53.727+0000 7f45ccf8b700 1 -- 172.21.15.45:0/2253008754 wait complete. 2024-04-14T15:04:54.211 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-04-14T15:04:55.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: Activating manager daemon y 2024-04-14T15:04:55.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: mgrmap e2: y(active, starting, since 0.00278337s) 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: Manager daemon y is now available 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' 2024-04-14T15:04:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:54 smithi045 ceph-mon[29624]: from='mgr.14100 v1:172.21.15.45:0/1957875348' entity='mgr.y' 2024-04-14T15:04:56.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:55 smithi045 ceph-mon[29624]: mgrmap e3: y(active, since 1.00706s) 2024-04-14T15:04:57.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:56 smithi045 ceph-mon[29624]: mgrmap e4: y(active, since 2s) 2024-04-14T15:04:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:04:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2019982352' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "1eba33fc-fa70-11ee-bc8e-c7b262605968", 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-14T15:05:03.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-14T15:05:03.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-14T15:05:03.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-14T15:05:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-14T15:04:40.069723+0000", 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.281+0000 7ff13f580700 1 Processor -- start 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.281+0000 7ff13f580700 1 -- start start 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff13f580700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff13807b2f0 con 0x7ff13807c6f0 2024-04-14T15:05:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff13d31c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff13807c6f0 0x7ff13807ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57494/0 (socket says 172.21.15.45:57494) 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff13d31c700 1 -- 172.21.15.45:0/754254621 learned_addr learned my addr 172.21.15.45:0/754254621 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129438700 0 0) 0x7ff13807b2f0 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1240035b0 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2988334429 0 0) 0x7ff1240035b0 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13807b4c0 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff128001580 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7ff128001d20 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.282+0000 7ff12ffff700 1 -- 172.21.15.45:0/754254621 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff128005960 con 0x7ff13807c6f0 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.283+0000 7ff13f580700 1 -- 172.21.15.45:0/754254621 >> v1:172.21.15.45:6789/0 conn(0x7ff13807c6f0 legacy=0x7ff13807ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.283+0000 7ff13f580700 1 -- 172.21.15.45:0/754254621 shutdown_connections 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.283+0000 7ff13f580700 1 -- 172.21.15.45:0/754254621 >> 172.21.15.45:0/754254621 conn(0x7ff138101e60 msgr2=0x7ff1381042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.283+0000 7ff13f580700 1 -- 172.21.15.45:0/754254621 shutdown_connections 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.283+0000 7ff13f580700 1 -- 172.21.15.45:0/754254621 wait complete. 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13f580700 1 Processor -- start 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13f580700 1 -- start start 2024-04-14T15:05:03.367 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13f580700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff13819cdf0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13d31c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff13807c6f0 0x7ff13819c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57510/0 (socket says 172.21.15.45:57510) 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13d31c700 1 -- 172.21.15.45:0/2019982352 learned_addr learned my addr 172.21.15.45:0/2019982352 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692159361 0 0) 0x7ff13819cdf0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff124005530 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1825903421 0 0) 0x7ff124005530 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff13819cdf0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff128004870 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3783773867 0 0) 0x7ff13819cdf0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13819cfc0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.284+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff13819d2b0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.285+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff1381a0e00 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.285+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7ff128001190 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.285+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff128005f70 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.285+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11c005150 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.285+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 4) ==== 46512+0+0 (unknown 3021500683 0 0) 0x7ff128011790 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.286+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7ff128047ee0 con 0x7ff13807c6f0 2024-04-14T15:05:03.368 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.291+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7ff128017660 con 0x7ff13807c6f0 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.588+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff11c005440 con 0x7ff13807c6f0 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.588+0000 7ff12e7fc700 1 -- 172.21.15.45:0/2019982352 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (unknown 4201413639 0 4196777611) 0x7ff128016f60 con 0x7ff13807c6f0 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 >> v1:172.21.15.45:6800/3894205139 conn(0x7ff12403a830 legacy=0x7ff12403ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 >> v1:172.21.15.45:6789/0 conn(0x7ff13807c6f0 legacy=0x7ff13819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 shutdown_connections 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 >> 172.21.15.45:0/2019982352 conn(0x7ff138101e60 msgr2=0x7ff13810adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 shutdown_connections 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:04:59.590+0000 7ff13f580700 1 -- 172.21.15.45:0/2019982352 wait complete. 2024-04-14T15:05:03.369 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-04-14T15:05:04.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1534531844' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-14T15:05:04.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.754+0000 7f57260b9700 1 Processor -- start 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.754+0000 7f57260b9700 1 -- start start 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f57260b9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5720079e50 con 0x7f5720107380 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5720107380 0x7f5720109770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34304/0 (socket says 172.21.15.45:34304) 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571f7fe700 1 -- 172.21.15.45:0/2264182600 learned_addr learned my addr 172.21.15.45:0/2264182600 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264121394 0 0) 0x7f5720079e50 con 0x7f5720107380 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57080035b0 con 0x7f5720107380 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3063530563 0 0) 0x7f57080035b0 con 0x7f5720107380 2024-04-14T15:05:04.464 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572010bea0 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.755+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f5710001580 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f5710001c50 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f571e7fc700 1 -- 172.21.15.45:0/2264182600 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f5710005960 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f57260b9700 1 -- 172.21.15.45:0/2264182600 >> v1:172.21.15.45:6789/0 conn(0x7f5720107380 legacy=0x7f5720109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f57260b9700 1 -- 172.21.15.45:0/2264182600 shutdown_connections 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f57260b9700 1 -- 172.21.15.45:0/2264182600 >> 172.21.15.45:0/2264182600 conn(0x7f5720100dd0 msgr2=0x7f57201031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f57260b9700 1 -- 172.21.15.45:0/2264182600 shutdown_connections 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.756+0000 7f57260b9700 1 -- 172.21.15.45:0/2264182600 wait complete. 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.757+0000 7f57260b9700 1 Processor -- start 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.757+0000 7f57260b9700 1 -- start start 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.757+0000 7f57260b9700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f572019ac80 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.757+0000 7f571f7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5720107380 0x7f572019a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34314/0 (socket says 172.21.15.45:34314) 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.757+0000 7f571f7fe700 1 -- 172.21.15.45:0/1534531844 learned_addr learned my addr 172.21.15.45:0/1534531844 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393447034 0 0) 0x7f572019ac80 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5708005580 con 0x7f5720107380 2024-04-14T15:05:04.465 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 142875560 0 0) 0x7f5708005580 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f572019ac80 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f5710001080 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 125937933 0 0) 0x7f572019ac80 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572019ae50 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f572019b140 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f572019ec90 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f5710001b90 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.758+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f5710005fb0 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.759+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 4) ==== 46512+0+0 (unknown 3021500683 0 0) 0x7f57100117d0 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.759+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f5710048d40 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.759+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570c005120 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.764+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f57100176c0 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.972+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f570c002400 con 0x7f5720107380 2024-04-14T15:05:04.466 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.973+0000 7f571cff9700 1 -- 172.21.15.45:0/1534531844 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+282 (unknown 1187553405 0 873977868) 0x7f5710016fc0 con 0x7f5720107380 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 >> v1:172.21.15.45:6800/3894205139 conn(0x7f570803a970 legacy=0x7f570803ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 >> v1:172.21.15.45:6789/0 conn(0x7f5720107380 legacy=0x7f572019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 shutdown_connections 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 >> 172.21.15.45:0/1534531844 conn(0x7f5720100dd0 msgr2=0x7f5720191200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 shutdown_connections 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:03.975+0000 7f57260b9700 1 -- 172.21.15.45:0/1534531844 wait complete. 2024-04-14T15:05:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-04-14T15:05:05.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1484975332' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-14T15:05:05.478 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:05 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setuser ceph since I am not root 2024-04-14T15:05:05.478 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:05 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setgroup ceph since I am not root 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.853+0000 7ff2fe93c700 1 Processor -- start 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.853+0000 7ff2fe93c700 1 -- start start 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.853+0000 7ff2fe93c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff2f807b310 con 0x7ff2f807c710 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2f807c710 0x7ff2f807ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34318/0 (socket says 172.21.15.45:34318) 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f7fff700 1 -- 172.21.15.45:0/466126242 learned_addr learned my addr 172.21.15.45:0/466126242 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 467605915 0 0) 0x7ff2f807b310 con 0x7ff2f807c710 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2e00035b0 con 0x7ff2f807c710 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 331129897 0 0) 0x7ff2e00035b0 con 0x7ff2f807c710 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f807b4e0 con 0x7ff2f807c710 2024-04-14T15:05:05.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff2e8001580 con 0x7ff2f807c710 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7ff2e8001c50 con 0x7ff2f807c710 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.854+0000 7ff2f6ffd700 1 -- 172.21.15.45:0/466126242 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff2e8005960 con 0x7ff2f807c710 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- 172.21.15.45:0/466126242 >> v1:172.21.15.45:6789/0 conn(0x7ff2f807c710 legacy=0x7ff2f807ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- 172.21.15.45:0/466126242 shutdown_connections 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- 172.21.15.45:0/466126242 >> 172.21.15.45:0/466126242 conn(0x7ff2f8101e20 msgr2=0x7ff2f8104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- 172.21.15.45:0/466126242 shutdown_connections 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- 172.21.15.45:0/466126242 wait complete. 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 Processor -- start 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.855+0000 7ff2fe93c700 1 -- start start 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2fe93c700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff2f819ce00 con 0x7ff2f807c710 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f7fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff2f807c710 0x7ff2f819c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34324/0 (socket says 172.21.15.45:34324) 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f7fff700 1 -- 172.21.15.45:0/1484975332 learned_addr learned my addr 172.21.15.45:0/1484975332 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115609709 0 0) 0x7ff2f819ce00 con 0x7ff2f807c710 2024-04-14T15:05:05.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2e0005580 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3253065288 0 0) 0x7ff2e0005580 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.856+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff2f819ce00 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff2e8001080 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3286686834 0 0) 0x7ff2f819ce00 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f819cfd0 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f819d2c0 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a0e10 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7ff2e8005920 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.857+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7ff2e8006090 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.858+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 4) ==== 46512+0+0 (unknown 3021500683 0 0) 0x7ff2e80117f0 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.858+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e4005150 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.858+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7ff2e8047e70 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:04.863+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7ff2e8017690 con 0x7ff2f807c710 2024-04-14T15:05:05.756 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.130+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff2e4005440 con 0x7ff2f807c710 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.181+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (unknown 2263024820 0 0) 0x7ff2e8016f90 con 0x7ff2f807c710 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.183+0000 7ff2f57fa700 1 -- 172.21.15.45:0/1484975332 <== mon.0 v1:172.21.15.45:6789/0 11 ==== mgrmap(e 5) ==== 46523+0+0 (unknown 3919204857 0 0) 0x7ff2e8046860 con 0x7ff2f807c710 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 >> v1:172.21.15.45:6800/3894205139 conn(0x7ff2e003a970 legacy=0x7ff2e003ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 >> v1:172.21.15.45:6789/0 conn(0x7ff2f807c710 legacy=0x7ff2f819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 shutdown_connections 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 >> 172.21.15.45:0/1484975332 conn(0x7ff2f8101e20 msgr2=0x7ff2f810ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 shutdown_connections 2024-04-14T15:05:05.757 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:05.184+0000 7ff2fe93c700 1 -- 172.21.15.45:0/1484975332 wait complete. 2024-04-14T15:05:05.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:05 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:05.755+0000 7f9d74c1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:05:05.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:05 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:05.856+0000 7f9d74c1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:05:06.420 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1484975332' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-14T15:05:06.420 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-mon[29624]: mgrmap e5: y(active, since 11s) 2024-04-14T15:05:06.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:06.532+0000 7f9d74c1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:05:06.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:06.627+0000 7f9d74c1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:05:07.187 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:06.772+0000 7f9d74c1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:05:07.187 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:06.871+0000 7f9d74c1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:05:07.187 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:06.957+0000 7f9d74c1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:05:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2208578320' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-14T15:05:07.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:07.227+0000 7f9d74c1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:05:07.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:07.340+0000 7f9d74c1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:05:07.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:07.429+0000 7f9d74c1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:05:07.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:05:07.518 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.213+0000 7f1d4d20d700 1 Processor -- start 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.213+0000 7f1d4d20d700 1 -- start start 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.213+0000 7f1d4d20d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1d48109be0 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.213+0000 7f1d46d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d481057b0 0x7f1d48105bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34366/0 (socket says 172.21.15.45:34366) 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.213+0000 7f1d46d9d700 1 -- 172.21.15.45:0/390568041 learned_addr learned my addr 172.21.15.45:0/390568041 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717636646 0 0) 0x7f1d48109be0 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d340035b0 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1215226775 0 0) 0x7f1d340035b0 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4810adc0 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1d30001580 con 0x7f1d481057b0 2024-04-14T15:05:07.519 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f1d30001d20 con 0x7f1d481057b0 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d45d9b700 1 -- 172.21.15.45:0/390568041 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1d30005960 con 0x7f1d481057b0 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d4d20d700 1 -- 172.21.15.45:0/390568041 >> v1:172.21.15.45:6789/0 conn(0x7f1d481057b0 legacy=0x7f1d48105bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d4d20d700 1 -- 172.21.15.45:0/390568041 shutdown_connections 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.214+0000 7f1d4d20d700 1 -- 172.21.15.45:0/390568041 >> 172.21.15.45:0/390568041 conn(0x7f1d48100d50 msgr2=0x7f1d48103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.215+0000 7f1d4d20d700 1 -- 172.21.15.45:0/390568041 shutdown_connections 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.215+0000 7f1d4d20d700 1 -- 172.21.15.45:0/390568041 wait complete. 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.215+0000 7f1d4d20d700 1 Processor -- start 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.215+0000 7f1d4d20d700 1 -- start start 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d4d20d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1d4819cd60 con 0x7f1d481057b0 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d46d9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d481057b0 0x7f1d4819c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34380/0 (socket says 172.21.15.45:34380) 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d46d9d700 1 -- 172.21.15.45:0/2208578320 learned_addr learned my addr 172.21.15.45:0/2208578320 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 363797828 0 0) 0x7f1d4819cd60 con 0x7f1d481057b0 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d34005530 con 0x7f1d481057b0 2024-04-14T15:05:07.520 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3792576411 0 0) 0x7f1d34005530 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1d4819cd60 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1d30004870 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2930017077 0 0) 0x7f1d4819cd60 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4819cf30 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4819d220 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.216+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1d481a0d70 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.217+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f1d30004a40 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.217+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1d300061f0 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.217+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) ==== 46523+0+0 (unknown 3919204857 0 0) 0x7f1d30011980 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.217+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d28005120 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.218+0000 7f1d4659c700 1 -- 172.21.15.45:0/2208578320 >> v1:172.21.15.45:6800/3894205139 conn(0x7f1d3403a8d0 legacy=0x7f1d3403cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.218+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f1d30047e30 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.222+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f1d300175c0 con 0x7f1d481057b0 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.419+0000 7f1d4659c700 1 -- 172.21.15.45:0/2208578320 >> v1:172.21.15.45:6800/3894205139 conn(0x7f1d3403a8d0 legacy=0x7f1d3403cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:07.521 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.492+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1d28005c20 con 0x7f1d481057b0 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.492+0000 7f1d3ffff700 1 -- 172.21.15.45:0/2208578320 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f1d30016ec0 con 0x7f1d481057b0 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.494+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 >> v1:172.21.15.45:6800/3894205139 conn(0x7f1d3403a8d0 legacy=0x7f1d3403cd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.494+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 >> v1:172.21.15.45:6789/0 conn(0x7f1d481057b0 legacy=0x7f1d4819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.494+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 shutdown_connections 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.494+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 >> 172.21.15.45:0/2208578320 conn(0x7f1d48100d50 msgr2=0x7f1d48103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.495+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 shutdown_connections 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:06.495+0000 7f1d4d20d700 1 -- 172.21.15.45:0/2208578320 wait complete. 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-14T15:05:07.522 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-04-14T15:05:07.813 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:07.528+0000 7f9d74c1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:05:07.813 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:07 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:07.712+0000 7f9d74c1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:05:08.840 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:08 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:08.705+0000 7f9d74c1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:05:09.179 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:08 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:08.840+0000 7f9d74c1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:05:09.472 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:09 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:09.178+0000 7f9d74c1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:05:09.472 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:09 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:09.270+0000 7f9d74c1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:05:09.472 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:09 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:09.472+0000 7f9d74c1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:05:09.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:09 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:09.662+0000 7f9d74c1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:05:10.216 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:09 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:09.954+0000 7f9d74c1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:05:10.217 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:10 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:10.216+0000 7f9d74c1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:05:10.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:10 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:10.461+0000 7f9d74c1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:05:11.726 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:11.291+0000 7f9d74c1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:05:11.977 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:11.877+0000 7f9d74c1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:05:11.977 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:11.977+0000 7f9d74c1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:05:12.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:12.200+0000 7f9d74c1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:05:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: Active manager daemon y restarted 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: Activating manager daemon y 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: osdmap e2: 0 total, 0 up, 0 in 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: mgrmap e6: y(active, starting, since 0.00334319s) 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: Manager daemon y is now available 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: Found migration_current of "None". Setting to last migration. 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:12.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:12.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:12 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: from='client.14124 v1:172.21.15.45:0/2351009673' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: mgrmap e7: y(active, since 1.00487s) 2024-04-14T15:05:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:13 smithi045 ceph-mon[29624]: from='client.14124 v1:172.21.15.45:0/2351009673' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-14T15:05:13.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2efcebf700 1 Processor -- start 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2efcebf700 1 -- start start 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2efcebf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2ef8079df0 con 0x7f2ef81051c0 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2ef659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ef81051c0 0x7f2ef81055e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34396/0 (socket says 172.21.15.45:34396) 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2ef659c700 1 -- 172.21.15.45:0/1002726734 learned_addr learned my addr 172.21.15.45:0/1002726734 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386550742 0 0) 0x7f2ef8079df0 con 0x7f2ef81051c0 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.914+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee00035b0 con 0x7f2ef81051c0 2024-04-14T15:05:13.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3468699372 0 0) 0x7f2ee00035b0 con 0x7f2ef81051c0 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef8106d00 con 0x7f2ef81051c0 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f2ee8001580 con 0x7f2ef81051c0 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f2ee8001c50 con 0x7f2ef81051c0 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2ef559a700 1 -- 172.21.15.45:0/1002726734 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f2ee8005960 con 0x7f2ef81051c0 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.915+0000 7f2efcebf700 1 -- 172.21.15.45:0/1002726734 >> v1:172.21.15.45:6789/0 conn(0x7f2ef81051c0 legacy=0x7f2ef81055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 -- 172.21.15.45:0/1002726734 shutdown_connections 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 -- 172.21.15.45:0/1002726734 >> 172.21.15.45:0/1002726734 conn(0x7f2ef8100d60 msgr2=0x7f2ef81031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 -- 172.21.15.45:0/1002726734 shutdown_connections 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 -- 172.21.15.45:0/1002726734 wait complete. 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 Processor -- start 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.916+0000 7f2efcebf700 1 -- start start 2024-04-14T15:05:13.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2efcebf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2ef819cea0 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ef659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ef81051c0 0x7f2ef819c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34404/0 (socket says 172.21.15.45:34404) 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ef659c700 1 -- 172.21.15.45:0/2351009673 learned_addr learned my addr 172.21.15.45:0/2351009673 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140212117 0 0) 0x7f2ef819cea0 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee0005580 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 91956065 0 0) 0x7f2ee0005580 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ef819cea0 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f2ee8001080 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3498037389 0 0) 0x7f2ef819cea0 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef819d070 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef819d360 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2ef81a0eb0 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.917+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f2ee8001c10 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.918+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f2ee8006030 con 0x7f2ef81051c0 2024-04-14T15:05:13.753 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.918+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 5) ==== 46523+0+0 (unknown 3919204857 0 0) 0x7f2ee80117a0 con 0x7f2ef81051c0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.918+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6800/3894205139 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ed8000d40 con 0x7f2ee003a9b0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.918+0000 7f2ef5d9b700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:07.918+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 2341496179 0 0) 0x7f2ee8048d50 con 0x7f2ef81051c0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:08.119+0000 7f2ef5d9b700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:08.520+0000 7f2ef5d9b700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:09.322+0000 7f2ef5d9b700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:10.923+0000 7f2ef5d9b700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/3894205139 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:12.204+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mgrmap(e 6) ==== 46325+0+0 (unknown 2891400997 0 0) 0x7f2ee8012d10 con 0x7f2ef81051c0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:12.204+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/3894205139 conn(0x7f2ee003a9b0 legacy=0x7f2ee003ce70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.204+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mgrmap(e 7) ==== 46417+0+0 (unknown 1342742735 0 0) 0x7f2ee8049770 con 0x7f2ef81051c0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.205+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6800/2598686815 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2ed8000d40 con 0x7f2ee003efd0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.206+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f2ed8000d40 con 0x7f2ee003efd0 2024-04-14T15:05:13.754 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.212+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 --> v1:172.21.15.45:6800/2598686815 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2ed8002840 con 0x7f2ee003efd0 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.212+0000 7f2ee77fe700 1 -- 172.21.15.45:0/2351009673 <== mgr.14120 v1:172.21.15.45:6800/2598686815 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (unknown 0 0 96372106) 0x7f2ed8002840 con 0x7f2ee003efd0 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6800/2598686815 conn(0x7f2ee003efd0 legacy=0x7f2ee00413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 >> v1:172.21.15.45:6789/0 conn(0x7f2ef81051c0 legacy=0x7f2ef819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 shutdown_connections 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 >> 172.21.15.45:0/2351009673 conn(0x7f2ef8100d60 msgr2=0x7f2ef8193370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 shutdown_connections 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:13.213+0000 7f2efcebf700 1 -- 172.21.15.45:0/2351009673 wait complete. 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-04-14T15:05:13.755 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-04-14T15:05:14.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:14 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:13] ENGINE Bus STARTING 2024-04-14T15:05:14.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:14 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:13] ENGINE Serving on http://172.21.15.45:8765 2024-04-14T15:05:14.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:14 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:13] ENGINE Serving on https://172.21.15.45:7150 2024-04-14T15:05:14.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:14 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:13] ENGINE Bus STARTED 2024-04-14T15:05:14.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:14 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.124+0000 7fda8ac09700 1 Processor -- start 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.124+0000 7fda8ac09700 1 -- start start 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.124+0000 7fda8ac09700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda8410ad30 con 0x7fda84106900 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.124+0000 7fda889a5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda84106900 0x7fda84106d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51548/0 (socket says 172.21.15.45:51548) 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.124+0000 7fda889a5700 1 -- 172.21.15.45:0/2951910182 learned_addr learned my addr 172.21.15.45:0/2951910182 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1854538073 0 0) 0x7fda8410ad30 con 0x7fda84106900 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda700035b0 con 0x7fda84106900 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178461380 0 0) 0x7fda700035b0 con 0x7fda84106900 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8410bf10 con 0x7fda84106900 2024-04-14T15:05:14.897 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda6c001580 con 0x7fda84106900 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fda6c001d20 con 0x7fda84106900 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda837fe700 1 -- 172.21.15.45:0/2951910182 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda6c005960 con 0x7fda84106900 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.125+0000 7fda8ac09700 1 -- 172.21.15.45:0/2951910182 >> v1:172.21.15.45:6789/0 conn(0x7fda84106900 legacy=0x7fda84106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- 172.21.15.45:0/2951910182 shutdown_connections 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- 172.21.15.45:0/2951910182 >> 172.21.15.45:0/2951910182 conn(0x7fda84101e80 msgr2=0x7fda841042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- 172.21.15.45:0/2951910182 shutdown_connections 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- 172.21.15.45:0/2951910182 wait complete. 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 Processor -- start 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- start start 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda8ac09700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda8419ab90 con 0x7fda84106900 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda889a5700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda84106900 0x7fda8419a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51554/0 (socket says 172.21.15.45:51554) 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.126+0000 7fda889a5700 1 -- 172.21.15.45:0/2638651513 learned_addr learned my addr 172.21.15.45:0/2638651513 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538387455 0 0) 0x7fda8419ab90 con 0x7fda84106900 2024-04-14T15:05:14.898 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda70005530 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4268211403 0 0) 0x7fda70005530 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fda8419ab90 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda6c004870 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1354716612 0 0) 0x7fda8419ab90 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8419ad60 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fda8419b050 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fda6c004a40 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda6c0062f0 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.127+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fda8419eba0 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.128+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 7) ==== 46417+0+0 (unknown 1342742735 0 0) 0x7fda6c011b60 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.128+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7fda6c047dc0 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.128+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda84064110 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.134+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fda6c0176d0 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.257+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7fda6c013130 con 0x7fda84106900 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.359+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fda8402e0b0 con 0x7fda7003a710 2024-04-14T15:05:14.899 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.364+0000 7fda81ffb700 1 -- 172.21.15.45:0/2638651513 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7fda8402e0b0 con 0x7fda7003a710 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 >> v1:172.21.15.45:6800/2598686815 conn(0x7fda7003a710 legacy=0x7fda7003cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 >> v1:172.21.15.45:6789/0 conn(0x7fda84106900 legacy=0x7fda8419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 shutdown_connections 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 >> 172.21.15.45:0/2638651513 conn(0x7fda84101e80 msgr2=0x7fda841042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 shutdown_connections 2024-04-14T15:05:14.900 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:14.367+0000 7fda8ac09700 1 -- 172.21.15.45:0/2638651513 wait complete. 2024-04-14T15:05:15.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:15 smithi045 ceph-mon[29624]: mgrmap e8: y(active, since 2s) 2024-04-14T15:05:15.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:15 smithi045 ceph-mon[29624]: from='client.14132 v1:172.21.15.45:0/2638651513' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:15.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:15 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:15.269 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:15 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:16.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-14T15:05:16.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.290+0000 7f787c31d700 1 Processor -- start 2024-04-14T15:05:16.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.290+0000 7f787c31d700 1 -- start start 2024-04-14T15:05:16.039 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f787c31d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f787407a700 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f787a0b9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7874107340 0x7f7874109730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51562/0 (socket says 172.21.15.45:51562) 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f787a0b9700 1 -- 172.21.15.45:0/3922462233 learned_addr learned my addr 172.21.15.45:0/3922462233 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3054029341 0 0) 0x7f787407a700 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78600035b0 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2756618696 0 0) 0x7f78600035b0 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787410be60 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7864001580 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f7864001c50 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.291+0000 7f78790b7700 1 -- 172.21.15.45:0/3922462233 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7864005960 con 0x7f7874107340 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.292+0000 7f787c31d700 1 -- 172.21.15.45:0/3922462233 >> v1:172.21.15.45:6789/0 conn(0x7f7874107340 legacy=0x7f7874109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.292+0000 7f787c31d700 1 -- 172.21.15.45:0/3922462233 shutdown_connections 2024-04-14T15:05:16.040 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.292+0000 7f787c31d700 1 -- 172.21.15.45:0/3922462233 >> 172.21.15.45:0/3922462233 conn(0x7f7874100d30 msgr2=0x7f7874103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.292+0000 7f787c31d700 1 -- 172.21.15.45:0/3922462233 shutdown_connections 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.292+0000 7f787c31d700 1 -- 172.21.15.45:0/3922462233 wait complete. 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f787c31d700 1 Processor -- start 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f787c31d700 1 -- start start 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f787c31d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f787419ac10 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f787a0b9700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7874107340 0x7f787419a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51572/0 (socket says 172.21.15.45:51572) 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f787a0b9700 1 -- 172.21.15.45:0/112004054 learned_addr learned my addr 172.21.15.45:0/112004054 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751597957 0 0) 0x7f787419ac10 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7860005580 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1753632654 0 0) 0x7f7860005580 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f787419ac10 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.293+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7864001080 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 685974998 0 0) 0x7f787419ac10 con 0x7f7874107340 2024-04-14T15:05:16.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787419ade0 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f787419b0d0 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f787419ec20 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f7864001bb0 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7864005ff0 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.294+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f7864006190 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.295+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7858005120 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.295+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f7864047e70 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.300+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f7864017650 con 0x7f7874107340 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.508+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7858000b90 con 0x7f786003aa00 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.509+0000 7f786b7fe700 1 -- 172.21.15.45:0/112004054 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (unknown 0 0 2070689548) 0x7f7858000b90 con 0x7f786003aa00 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 >> v1:172.21.15.45:6800/2598686815 conn(0x7f786003aa00 legacy=0x7f786003cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 >> v1:172.21.15.45:6789/0 conn(0x7f7874107340 legacy=0x7f787419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:16.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 shutdown_connections 2024-04-14T15:05:16.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 >> 172.21.15.45:0/112004054 conn(0x7f7874100d30 msgr2=0x7f7874191180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:16.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 shutdown_connections 2024-04-14T15:05:16.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:15.511+0000 7f787c31d700 1 -- 172.21.15.45:0/112004054 wait complete. 2024-04-14T15:05:16.043 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-04-14T15:05:16.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-mon[29624]: from='client.14134 v1:172.21.15.45:0/112004054' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: Generating public/private rsa key pair. 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: Your identification has been saved in /tmp/tmppjbkfuyo/key. 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: Your public key has been saved in /tmp/tmppjbkfuyo/key.pub. 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: The key fingerprint is: 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: SHA256:RoiGqBGzIVI+tYGBikEcU9wxQIHgv9DCvQjA2hp0A18 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: The key's randomart image is: 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: +---[RSA 3072]----+ 2024-04-14T15:05:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |@BOB*E. | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |BX=+.=.. | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |B=+++ . . | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |Oo=o. . | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |=+.+ S | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |.o+ o . | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: |.. o | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: | | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: | | 2024-04-14T15:05:16.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: +----[SHA256]-----+ 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6957b87700 1 Processor -- start 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6957b87700 1 -- start start 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6957b87700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6950109be0 con 0x7f69501057b0 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6955923700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f69501057b0 0x7f6950105bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51584/0 (socket says 172.21.15.45:51584) 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6955923700 1 -- 172.21.15.45:0/1067151905 learned_addr learned my addr 172.21.15.45:0/1067151905 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806224224 0 0) 0x7f6950109be0 con 0x7f69501057b0 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.465+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f693c0035b0 con 0x7f69501057b0 2024-04-14T15:05:17.486 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137753591 0 0) 0x7f693c0035b0 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695010adc0 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f694c001580 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f694c001c50 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6954921700 1 -- 172.21.15.45:0/1067151905 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f694c005960 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.466+0000 7f6957b87700 1 -- 172.21.15.45:0/1067151905 >> v1:172.21.15.45:6789/0 conn(0x7f69501057b0 legacy=0x7f6950105bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- 172.21.15.45:0/1067151905 shutdown_connections 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- 172.21.15.45:0/1067151905 >> 172.21.15.45:0/1067151905 conn(0x7f6950100d50 msgr2=0x7f6950103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- 172.21.15.45:0/1067151905 shutdown_connections 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- 172.21.15.45:0/1067151905 wait complete. 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 Processor -- start 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- start start 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.467+0000 7f6957b87700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f695019abb0 con 0x7f69501057b0 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6955923700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f69501057b0 0x7f695019a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51588/0 (socket says 172.21.15.45:51588) 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6955923700 1 -- 172.21.15.45:0/3167402142 learned_addr learned my addr 172.21.15.45:0/3167402142 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:17.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481536613 0 0) 0x7f695019abb0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f693c005580 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1085380084 0 0) 0x7f693c005580 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f695019abb0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f694c001080 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 932554820 0 0) 0x7f695019abb0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695019ad80 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f695019b070 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f695019ebc0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f694c005900 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.468+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f694c006220 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.469+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f694c0074f0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.469+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6934005150 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.470+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f694c047f30 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.474+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f694c0175f0 con 0x7f69501057b0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.682+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6934000bc0 con 0x7f693c03aab0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.875+0000 7f6946ffd700 1 -- 172.21.15.45:0/3167402142 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7f6934000bc0 con 0x7f693c03aab0 2024-04-14T15:05:17.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 >> v1:172.21.15.45:6800/2598686815 conn(0x7f693c03aab0 legacy=0x7f693c03cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:17.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 >> v1:172.21.15.45:6789/0 conn(0x7f69501057b0 legacy=0x7f695019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:17.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 shutdown_connections 2024-04-14T15:05:17.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 >> 172.21.15.45:0/3167402142 conn(0x7f6950100d50 msgr2=0x7f6950079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:17.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 shutdown_connections 2024-04-14T15:05:17.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:16.877+0000 7f6957b87700 1 -- 172.21.15.45:0/3167402142 wait complete. 2024-04-14T15:05:18.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:17 smithi045 ceph-mon[29624]: from='client.14136 v1:172.21.15.45:0/3167402142' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:18.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:17 smithi045 ceph-mon[29624]: Generating ssh key... 2024-04-14T15:05:18.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:17 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:18.095 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:17 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:18.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXgSdPPFEAubTxmLRuP3NCCYlNy6O2QrkybXwEvH7TG5D6rTswr0pZTR4W5IitUuro60BqsAMuUy2lzLoMKiF9z2GNchM16VDu5Lvfs6M8qMHxtQ9N/Nhk7EE0cTFGgYpfkTT/3+mJfKg7956W5tmm560/lo+e9VFEWsvQVqYA0EZ3sNKe/SpS5XvyxVZa7KJaSh0wao7IkeTsVK7uigMUdb0UiiJ9qkZlCoYS5YiO+XZZ/VOEAzrBdZnSniSEaQUwrYlVk9Lqc4n69l5vj2odN7IVAZEfjYAJboL+xPnZt89JZ+VSBLFaO4GFNKJlg0Txb+JJjpa9byrONce510BChU9Ba8qcQd3GhWeQClKWVt6quASOsrn5ibUokkRB984FUBQ6EWvCWvy+Pr0dLvKtvmfsDTSsFF5tmMh+OGgk59MxEbPyQucdUJpURWis6sKdzqXNdkItokSL0065T4IK6dbQa6pV8HeYlzvm9OucYTzI9GIEgb6BJ2ZvPnpaYcM= ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:05:18.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca5e754700 1 Processor -- start 2024-04-14T15:05:18.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca5e754700 1 -- start start 2024-04-14T15:05:18.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca5e754700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fca5810ace0 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca57fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fca581068b0 0x7fca58106cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51592/0 (socket says 172.21.15.45:51592) 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca57fff700 1 -- 172.21.15.45:0/2501799051 learned_addr learned my addr 172.21.15.45:0/2501799051 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.879+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491536216 0 0) 0x7fca5810ace0 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca400035b0 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 720253312 0 0) 0x7fca400035b0 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5810bec0 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fca48001580 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fca48001d20 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca56ffd700 1 -- 172.21.15.45:0/2501799051 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fca48005960 con 0x7fca581068b0 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca5e754700 1 -- 172.21.15.45:0/2501799051 >> v1:172.21.15.45:6789/0 conn(0x7fca581068b0 legacy=0x7fca58106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca5e754700 1 -- 172.21.15.45:0/2501799051 shutdown_connections 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca5e754700 1 -- 172.21.15.45:0/2501799051 >> 172.21.15.45:0/2501799051 conn(0x7fca58101e50 msgr2=0x7fca58104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca5e754700 1 -- 172.21.15.45:0/2501799051 shutdown_connections 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.880+0000 7fca5e754700 1 -- 172.21.15.45:0/2501799051 wait complete. 2024-04-14T15:05:18.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.881+0000 7fca5e754700 1 Processor -- start 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.881+0000 7fca5e754700 1 -- start start 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.881+0000 7fca5e754700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fca5819ab20 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca57fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fca581068b0 0x7fca5819a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51600/0 (socket says 172.21.15.45:51600) 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca57fff700 1 -- 172.21.15.45:0/1474086079 learned_addr learned my addr 172.21.15.45:0/1474086079 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881760988 0 0) 0x7fca5819ab20 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca40005530 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2069735895 0 0) 0x7fca40005530 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca5819ab20 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fca48004870 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 214565462 0 0) 0x7fca5819ab20 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5819acf0 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fca5819afe0 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fca5819eb30 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7fca48001d20 con 0x7fca581068b0 2024-04-14T15:05:18.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.882+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fca48006010 con 0x7fca581068b0 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.883+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7fca48011840 con 0x7fca581068b0 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.883+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca44005150 con 0x7fca581068b0 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.883+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7fca48048070 con 0x7fca581068b0 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:17.888+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fca48017760 con 0x7fca581068b0 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.094+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fca44000bc0 con 0x7fca4003a830 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.095+0000 7fca557fa700 1 -- 172.21.15.45:0/1474086079 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (unknown 0 0 3816703815) 0x7fca44000bc0 con 0x7fca4003a830 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.097+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 >> v1:172.21.15.45:6800/2598686815 conn(0x7fca4003a830 legacy=0x7fca4003ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.097+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 >> v1:172.21.15.45:6789/0 conn(0x7fca581068b0 legacy=0x7fca5819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.097+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 shutdown_connections 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.097+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 >> 172.21.15.45:0/1474086079 conn(0x7fca58101e50 msgr2=0x7fca58104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.097+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 shutdown_connections 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:18.098+0000 7fca5e754700 1 -- 172.21.15.45:0/1474086079 wait complete. 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-04-14T15:05:18.671 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-04-14T15:05:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:18 smithi045 ceph-mon[29624]: from='client.14138 v1:172.21.15.45:0/1474086079' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:20.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:20 smithi045 ceph-mon[29624]: from='client.14140 v1:172.21.15.45:0/3799624054' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:20.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:20 smithi045 ceph-mon[29624]: Deploying cephadm binary to smithi045 2024-04-14T15:05:24.233 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ffc8c5700 1 Processor -- start 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ffc8c5700 1 -- start start 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ffc8c5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0ff807a6e0 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ff659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ff8107360 0x7f0ff8109750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51616/0 (socket says 172.21.15.45:51616) 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ff659c700 1 -- 172.21.15.45:0/2882621647 learned_addr learned my addr 172.21.15.45:0/2882621647 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862103347 0 0) 0x7f0ff807a6e0 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.066+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe00035b0 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2882939681 0 0) 0x7f0fe00035b0 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff810be80 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0fe8001580 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f0fe8001d20 con 0x7f0ff8107360 2024-04-14T15:05:24.234 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ff559a700 1 -- 172.21.15.45:0/2882621647 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0fe8005960 con 0x7f0ff8107360 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/2882621647 >> v1:172.21.15.45:6789/0 conn(0x7f0ff8107360 legacy=0x7f0ff8109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/2882621647 shutdown_connections 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.067+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/2882621647 >> 172.21.15.45:0/2882621647 conn(0x7f0ff8100d50 msgr2=0x7f0ff8103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.068+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/2882621647 shutdown_connections 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.068+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/2882621647 wait complete. 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.068+0000 7f0ffc8c5700 1 Processor -- start 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.068+0000 7f0ffc8c5700 1 -- start start 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.068+0000 7f0ffc8c5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0ff819ab80 con 0x7f0ff8107360 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0ff659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0ff8107360 0x7f0ff819a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51628/0 (socket says 172.21.15.45:51628) 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0ff659c700 1 -- 172.21.15.45:0/3799624054 learned_addr learned my addr 172.21.15.45:0/3799624054 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822772779 0 0) 0x7f0ff819ab80 con 0x7f0ff8107360 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe0005530 con 0x7f0ff8107360 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1850110986 0 0) 0x7f0fe0005530 con 0x7f0ff8107360 2024-04-14T15:05:24.235 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ff819ab80 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0fe8004870 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3697756612 0 0) 0x7f0ff819ab80 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff819ad50 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff819b040 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0ff819eb90 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f0fe8001190 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.069+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0fe8005f60 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.070+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f0fe80116d0 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.070+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd8005150 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.070+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f0fe8047f90 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.075+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0fe8017600 con 0x7f0ff8107360 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:19.290+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7f0fd8000bc0 con 0x7f0fe003a8d0 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.636+0000 7f0fe77fe700 1 -- 172.21.15.45:0/3799624054 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (unknown 0 0 35162163) 0x7f0fd8000bc0 con 0x7f0fe003a8d0 2024-04-14T15:05:24.236 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.639+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 >> v1:172.21.15.45:6800/2598686815 conn(0x7f0fe003a8d0 legacy=0x7f0fe003cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.639+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 >> v1:172.21.15.45:6789/0 conn(0x7f0ff8107360 legacy=0x7f0ff819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.639+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 shutdown_connections 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.639+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 >> 172.21.15.45:0/3799624054 conn(0x7f0ff8100d50 msgr2=0x7f0ff81911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.640+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 shutdown_connections 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:23.640+0000 7f0ffc8c5700 1 -- 172.21.15.45:0/3799624054 wait complete. 2024-04-14T15:05:24.237 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-04-14T15:05:24.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:24 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:24.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:24 smithi045 ceph-mon[29624]: Added host smithi045 2024-04-14T15:05:24.889 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:24 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.657+0000 7f8e325f6700 1 Processor -- start 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.657+0000 7f8e325f6700 1 -- start start 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.657+0000 7f8e325f6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8e2c109c20 con 0x7f8e2c105cd0 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.657+0000 7f8e2bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e2c105cd0 0x7f8e2c1060f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34650/0 (socket says 172.21.15.45:34650) 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.657+0000 7f8e2bfff700 1 -- 172.21.15.45:0/1893855018 learned_addr learned my addr 172.21.15.45:0/1893855018 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252323575 0 0) 0x7f8e2c109c20 con 0x7f8e2c105cd0 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e140035b0 con 0x7f8e2c105cd0 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2923900133 0 0) 0x7f8e140035b0 con 0x7f8e2c105cd0 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c10ae00 con 0x7f8e2c105cd0 2024-04-14T15:05:25.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8e1c001580 con 0x7f8e2c105cd0 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f8e1c001c50 con 0x7f8e2c105cd0 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.658+0000 7f8e2affd700 1 -- 172.21.15.45:0/1893855018 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8e1c005960 con 0x7f8e2c105cd0 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- 172.21.15.45:0/1893855018 >> v1:172.21.15.45:6789/0 conn(0x7f8e2c105cd0 legacy=0x7f8e2c1060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- 172.21.15.45:0/1893855018 shutdown_connections 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- 172.21.15.45:0/1893855018 >> 172.21.15.45:0/1893855018 conn(0x7f8e2c100550 msgr2=0x7f8e2c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- 172.21.15.45:0/1893855018 shutdown_connections 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- 172.21.15.45:0/1893855018 wait complete. 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 Processor -- start 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.659+0000 7f8e325f6700 1 -- start start 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e325f6700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8e2c19ac70 con 0x7f8e2c105cd0 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e2bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e2c105cd0 0x7f8e2c19a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34652/0 (socket says 172.21.15.45:34652) 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e2bfff700 1 -- 172.21.15.45:0/1573011633 learned_addr learned my addr 172.21.15.45:0/1573011633 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219161630 0 0) 0x7f8e2c19ac70 con 0x7f8e2c105cd0 2024-04-14T15:05:25.670 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e14005580 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2992341000 0 0) 0x7f8e14005580 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e2c19ac70 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8e1c001080 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.660+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1293380062 0 0) 0x7f8e2c19ac70 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.661+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c19ae40 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.661+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c19b130 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.661+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8e2c19ec80 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.661+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f8e1c001b90 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.661+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8e1c005fb0 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.662+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f8e1c011720 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.662+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e18005150 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.662+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f8e1c047eb0 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.667+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f8e1c017650 con 0x7f8e2c105cd0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.889+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8e18000bc0 con 0x7f8e1403a9c0 2024-04-14T15:05:25.671 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.893+0000 7f8e297fa700 1 -- 172.21.15.45:0/1573011633 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7f8e18000bc0 con 0x7f8e1403a9c0 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 >> v1:172.21.15.45:6800/2598686815 conn(0x7f8e1403a9c0 legacy=0x7f8e1403ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 >> v1:172.21.15.45:6789/0 conn(0x7f8e2c105cd0 legacy=0x7f8e2c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 shutdown_connections 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 >> 172.21.15.45:0/1573011633 conn(0x7f8e2c100550 msgr2=0x7f8e2c104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 shutdown_connections 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:24.895+0000 7f8e325f6700 1 -- 172.21.15.45:0/1573011633 wait complete. 2024-04-14T15:05:25.672 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-04-14T15:05:25.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:25 smithi045 ceph-mon[29624]: from='client.14142 v1:172.21.15.45:0/1573011633' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:25.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:25 smithi045 ceph-mon[29624]: Saving service mon spec with placement count:5 2024-04-14T15:05:25.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:25 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.085+0000 7f1454233700 1 Processor -- start 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.085+0000 7f1454233700 1 -- start start 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.085+0000 7f1454233700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f144c109be0 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.085+0000 7f1451fcf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f144c1057b0 0x7f144c105bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34668/0 (socket says 172.21.15.45:34668) 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.085+0000 7f1451fcf700 1 -- 172.21.15.45:0/2404344336 learned_addr learned my addr 172.21.15.45:0/2404344336 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2657385043 0 0) 0x7f144c109be0 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14380057c0 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3861679149 0 0) 0x7f14380057c0 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c10adc0 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1448001580 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f1448001c50 con 0x7f144c1057b0 2024-04-14T15:05:27.223 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1450fcd700 1 -- 172.21.15.45:0/2404344336 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1448005960 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.086+0000 7f1454233700 1 -- 172.21.15.45:0/2404344336 >> v1:172.21.15.45:6789/0 conn(0x7f144c1057b0 legacy=0x7f144c105bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- 172.21.15.45:0/2404344336 shutdown_connections 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- 172.21.15.45:0/2404344336 >> 172.21.15.45:0/2404344336 conn(0x7f144c100d50 msgr2=0x7f144c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- 172.21.15.45:0/2404344336 shutdown_connections 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- 172.21.15.45:0/2404344336 wait complete. 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 Processor -- start 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- start start 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.087+0000 7f1454233700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f144c19cdc0 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1451fcf700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f144c1057b0 0x7f144c19c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34678/0 (socket says 172.21.15.45:34678) 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1451fcf700 1 -- 172.21.15.45:0/1392990915 learned_addr learned my addr 172.21.15.45:0/1392990915 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3466958093 0 0) 0x7f144c19cdc0 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1438007870 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1183873412 0 0) 0x7f1438007870 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f144c19cdc0 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1448001080 con 0x7f144c1057b0 2024-04-14T15:05:27.224 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3034437323 0 0) 0x7f144c19cdc0 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c19cf90 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f144c19d280 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f144c1a0dd0 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1037+0+0 (unknown 2512772806 0 0) 0x7f1448005580 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.088+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1448005fd0 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.089+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430005120 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.089+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f1448011740 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.089+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f1448047f80 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.094+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f1448017690 con 0x7f144c1057b0 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.319+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 --> v1:172.21.15.45:6800/2598686815 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1430000b90 con 0x7f143803aa10 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.324+0000 7f1442ffd700 1 -- 172.21.15.45:0/1392990915 <== mgr.14120 v1:172.21.15.45:6800/2598686815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7f1430000b90 con 0x7f143803aa10 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 >> v1:172.21.15.45:6800/2598686815 conn(0x7f143803aa10 legacy=0x7f143803ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 >> v1:172.21.15.45:6789/0 conn(0x7f144c1057b0 legacy=0x7f144c19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 shutdown_connections 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 >> 172.21.15.45:0/1392990915 conn(0x7f144c100d50 msgr2=0x7f144c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:27.225 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 shutdown_connections 2024-04-14T15:05:27.226 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:26.326+0000 7f1454233700 1 -- 172.21.15.45:0/1392990915 wait complete. 2024-04-14T15:05:27.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:27 smithi045 ceph-mon[29624]: from='client.14144 v1:172.21.15.45:0/1392990915' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:27.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:27 smithi045 ceph-mon[29624]: Saving service mgr spec with placement count:2 2024-04-14T15:05:27.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:27 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:27.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:27 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.650+0000 7fc025709700 1 Processor -- start 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc025709700 1 -- start start 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc025709700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc020079e50 con 0x7fc020107380 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc020107380 0x7fc020109770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34690/0 (socket says 172.21.15.45:34690) 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01effd700 1 -- 172.21.15.45:0/1306496262 learned_addr learned my addr 172.21.15.45:0/1306496262 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918206436 0 0) 0x7fc020079e50 con 0x7fc020107380 2024-04-14T15:05:28.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0080035b0 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3755800761 0 0) 0x7fc0080035b0 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02010bea0 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.651+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc014001580 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fc014001c70 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc01dffb700 1 -- 172.21.15.45:0/1306496262 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc014005960 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc025709700 1 -- 172.21.15.45:0/1306496262 >> v1:172.21.15.45:6789/0 conn(0x7fc020107380 legacy=0x7fc020109770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc025709700 1 -- 172.21.15.45:0/1306496262 shutdown_connections 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc025709700 1 -- 172.21.15.45:0/1306496262 >> 172.21.15.45:0/1306496262 conn(0x7fc020100dd0 msgr2=0x7fc0201031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc025709700 1 -- 172.21.15.45:0/1306496262 shutdown_connections 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.652+0000 7fc025709700 1 -- 172.21.15.45:0/1306496262 wait complete. 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.653+0000 7fc025709700 1 Processor -- start 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.653+0000 7fc025709700 1 -- start start 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.653+0000 7fc025709700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc02019ac80 con 0x7fc020107380 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.653+0000 7fc01effd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc020107380 0x7fc02019a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34696/0 (socket says 172.21.15.45:34696) 2024-04-14T15:05:28.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.653+0000 7fc01effd700 1 -- 172.21.15.45:0/1039180013 learned_addr learned my addr 172.21.15.45:0/1039180013 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4176519127 0 0) 0x7fc02019ac80 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc008005580 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1950651834 0 0) 0x7fc008005580 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc02019ac80 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc014001080 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1960396033 0 0) 0x7fc02019ac80 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02019ae50 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc02019b140 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc02019ec90 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fc014004a60 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.654+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fc0140062e0 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.655+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7fc014011a50 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.655+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc020064110 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.655+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7fc014048de0 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.660+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fc0140176b0 con 0x7fc020107380 2024-04-14T15:05:28.498 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.867+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fc02002e040 con 0x7fc020107380 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.869+0000 7fc007fff700 1 -- 172.21.15.45:0/1039180013 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (unknown 2684808024 0 0) 0x7fc014016fb0 con 0x7fc020107380 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.871+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 >> v1:172.21.15.45:6800/2598686815 conn(0x7fc00803aa80 legacy=0x7fc00803cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.871+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 >> v1:172.21.15.45:6789/0 conn(0x7fc020107380 legacy=0x7fc02019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.872+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 shutdown_connections 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.872+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 >> 172.21.15.45:0/1039180013 conn(0x7fc020100dd0 msgr2=0x7fc0201911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.872+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 shutdown_connections 2024-04-14T15:05:28.499 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:27.872+0000 7fc025709700 1 -- 172.21.15.45:0/1039180013 wait complete. 2024-04-14T15:05:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:28 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1039180013' entity='client.admin' 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.907+0000 7fda32233700 1 Processor -- start 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda32233700 1 -- start start 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda32233700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda2c07a700 con 0x7fda2c107340 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda2b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda2c107340 0x7fda2c109730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34704/0 (socket says 172.21.15.45:34704) 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda2b7fe700 1 -- 172.21.15.45:0/834623508 learned_addr learned my addr 172.21.15.45:0/834623508 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3126762729 0 0) 0x7fda2c07a700 con 0x7fda2c107340 2024-04-14T15:05:29.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda140035b0 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.908+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 822104987 0 0) 0x7fda140035b0 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c10be60 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda1c001580 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fda1c004930 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda2a7fc700 1 -- 172.21.15.45:0/834623508 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda1c005a70 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda32233700 1 -- 172.21.15.45:0/834623508 >> v1:172.21.15.45:6789/0 conn(0x7fda2c107340 legacy=0x7fda2c109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda32233700 1 -- 172.21.15.45:0/834623508 shutdown_connections 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.909+0000 7fda32233700 1 -- 172.21.15.45:0/834623508 >> 172.21.15.45:0/834623508 conn(0x7fda2c100d50 msgr2=0x7fda2c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.910+0000 7fda32233700 1 -- 172.21.15.45:0/834623508 shutdown_connections 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.910+0000 7fda32233700 1 -- 172.21.15.45:0/834623508 wait complete. 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.910+0000 7fda32233700 1 Processor -- start 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.910+0000 7fda32233700 1 -- start start 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.910+0000 7fda32233700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda2c1a04f0 con 0x7fda2c107340 2024-04-14T15:05:29.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda2b7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fda2c107340 0x7fda2c106bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34710/0 (socket says 172.21.15.45:34710) 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda2b7fe700 1 -- 172.21.15.45:0/2352384486 learned_addr learned my addr 172.21.15.45:0/2352384486 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223879727 0 0) 0x7fda2c1a04f0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda14005530 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2477372994 0 0) 0x7fda14005530 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fda2c1a04f0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda1c001080 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 10197493 0 0) 0x7fda2c1a04f0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1a16d0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1a0720 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.911+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fda2c1a0c60 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.912+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fda1c004930 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.912+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fda1c006020 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.912+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7fda1c0072f0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.912+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda18005150 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.913+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7fda1c048e00 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:28.921+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fda1c0176d0 con 0x7fda2c107340 2024-04-14T15:05:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.142+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fda18005440 con 0x7fda2c107340 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.144+0000 7fda28ff9700 1 -- 172.21.15.45:0/2352384486 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (unknown 1496729260 0 0) 0x7fda1c016fd0 con 0x7fda2c107340 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.146+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 >> v1:172.21.15.45:6800/2598686815 conn(0x7fda1403aa60 legacy=0x7fda1403cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.146+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 >> v1:172.21.15.45:6789/0 conn(0x7fda2c107340 legacy=0x7fda2c106bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.146+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 shutdown_connections 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.146+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 >> 172.21.15.45:0/2352384486 conn(0x7fda2c100d50 msgr2=0x7fda2c1911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.147+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 shutdown_connections 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:29.147+0000 7fda32233700 1 -- 172.21.15.45:0/2352384486 wait complete. 2024-04-14T15:05:29.908 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-04-14T15:05:30.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2352384486' entity='client.admin' 2024-04-14T15:05:31.739 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:31 smithi045 ceph-mon[29624]: from='mgr.14120 v1:172.21.15.45:0/3647192266' entity='mgr.y' 2024-04-14T15:05:31.739 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/381486802' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-14T15:05:31.739 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:31 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setuser ceph since I am not root 2024-04-14T15:05:31.739 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:31 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setgroup ceph since I am not root 2024-04-14T15:05:32.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f708e9ad700 1 Processor -- start 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f708e9ad700 1 -- start start 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f708e9ad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f708810ad10 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f7087fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f70881068e0 0x7f7088106d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34718/0 (socket says 172.21.15.45:34718) 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f7087fff700 1 -- 172.21.15.45:0/2808687535 learned_addr learned my addr 172.21.15.45:0/2808687535 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662708437 0 0) 0x7f708810ad10 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.340+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70700035b0 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1894427519 0 0) 0x7f70700035b0 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708810bef0 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7078001580 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f7078004930 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f7086ffd700 1 -- 172.21.15.45:0/2808687535 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7078005960 con 0x7f70881068e0 2024-04-14T15:05:32.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f708e9ad700 1 -- 172.21.15.45:0/2808687535 >> v1:172.21.15.45:6789/0 conn(0x7f70881068e0 legacy=0x7f7088106d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f708e9ad700 1 -- 172.21.15.45:0/2808687535 shutdown_connections 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f708e9ad700 1 -- 172.21.15.45:0/2808687535 >> 172.21.15.45:0/2808687535 conn(0x7f7088101e60 msgr2=0x7f70881042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f708e9ad700 1 -- 172.21.15.45:0/2808687535 shutdown_connections 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.341+0000 7f708e9ad700 1 -- 172.21.15.45:0/2808687535 wait complete. 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.342+0000 7f708e9ad700 1 Processor -- start 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.342+0000 7f708e9ad700 1 -- start start 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.342+0000 7f708e9ad700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f708819ab70 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.342+0000 7f7087fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f70881068e0 0x7f708819a460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34730/0 (socket says 172.21.15.45:34730) 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.342+0000 7f7087fff700 1 -- 172.21.15.45:0/381486802 learned_addr learned my addr 172.21.15.45:0/381486802 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205273845 0 0) 0x7f708819ab70 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7070005530 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 209992474 0 0) 0x7f7070005530 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f708819ab70 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7078001080 con 0x7f70881068e0 2024-04-14T15:05:32.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 486314863 0 0) 0x7f708819ab70 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708819ad40 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f708819b030 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f708819eb80 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f7078004930 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.343+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f7078006030 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.344+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 8) ==== 46523+0+0 (unknown 1702799921 0 0) 0x7f70780118a0 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.344+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7074005150 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.344+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f7078048060 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.349+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f70780177a0 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:30.625+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7074005440 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.408+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (unknown 1498667528 0 0) 0x7f70780170a0 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.410+0000 7f70857fa700 1 -- 172.21.15.45:0/381486802 <== mon.0 v1:172.21.15.45:6789/0 11 ==== mgrmap(e 9) ==== 46536+0+0 (unknown 1243592525 0 0) 0x7f70780469a0 con 0x7f70881068e0 2024-04-14T15:05:32.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 >> v1:172.21.15.45:6800/2598686815 conn(0x7f707003a9d0 legacy=0x7f707003ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:32.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 >> v1:172.21.15.45:6789/0 conn(0x7f70881068e0 legacy=0x7f708819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:32.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 shutdown_connections 2024-04-14T15:05:32.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 >> 172.21.15.45:0/381486802 conn(0x7f7088101e60 msgr2=0x7f70881042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:32.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 shutdown_connections 2024-04-14T15:05:32.025 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:31.412+0000 7f708e9ad700 1 -- 172.21.15.45:0/381486802 wait complete. 2024-04-14T15:05:32.269 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:32.016+0000 7f4f61895200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:05:32.269 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:32.129+0000 7f4f61895200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:05:32.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/381486802' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-14T15:05:32.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-mon[29624]: mgrmap e9: y(active, since 19s) 2024-04-14T15:05:32.918 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:32.822+0000 7f4f61895200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:05:32.918 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:32 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:32.917+0000 7f4f61895200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:05:33.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.066+0000 7f4f61895200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:05:33.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.164+0000 7f4f61895200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:05:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:05:33.353 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.451+0000 7f93a4f7d700 1 Processor -- start 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.451+0000 7f93a4f7d700 1 -- start start 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f93a4f7d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f93a010ae60 con 0x7f93a0104bb0 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939e59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f93a0104bb0 0x7f93a0104fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34064/0 (socket says 172.21.15.45:34064) 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939e59c700 1 -- 172.21.15.45:0/2900747077 learned_addr learned my addr 172.21.15.45:0/2900747077 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660647960 0 0) 0x7f93a010ae60 con 0x7f93a0104bb0 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f938c0035b0 con 0x7f93a0104bb0 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505302356 0 0) 0x7f938c0035b0 con 0x7f93a0104bb0 2024-04-14T15:05:33.354 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a010c040 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9390001580 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9390001c70 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.452+0000 7f939d59a700 1 -- 172.21.15.45:0/2900747077 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9390005960 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.453+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2900747077 >> v1:172.21.15.45:6789/0 conn(0x7f93a0104bb0 legacy=0x7f93a0104fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.453+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2900747077 shutdown_connections 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.453+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2900747077 >> 172.21.15.45:0/2900747077 conn(0x7f93a0100550 msgr2=0x7f93a0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.453+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2900747077 shutdown_connections 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.453+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2900747077 wait complete. 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.454+0000 7f93a4f7d700 1 Processor -- start 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.454+0000 7f93a4f7d700 1 -- start start 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.454+0000 7f93a4f7d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f93a019ac70 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f939e59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f93a0104bb0 0x7f93a019a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34072/0 (socket says 172.21.15.45:34072) 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f939e59c700 1 -- 172.21.15.45:0/2828327321 learned_addr learned my addr 172.21.15.45:0/2828327321 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204963968 0 0) 0x7f93a019ac70 con 0x7f93a0104bb0 2024-04-14T15:05:33.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f938c005580 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3296738716 0 0) 0x7f938c005580 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f93a019ac70 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9390005920 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2589976402 0 0) 0x7f93a019ac70 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a019ae40 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f93a019b130 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.455+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f93a019ec80 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.456+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9390005660 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.456+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f9390005f70 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.456+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 9) ==== 46536+0+0 (unknown 1243592525 0 0) 0x7f93900117a0 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.456+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7f9390048ea0 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.456+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9380005150 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.457+0000 7f939dd9b700 1 -- 172.21.15.45:0/2828327321 >> v1:172.21.15.45:6800/2598686815 conn(0x7f938c03a8c0 legacy=0x7f938c03cd80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.461+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9390017710 con 0x7f93a0104bb0 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.658+0000 7f939dd9b700 1 -- 172.21.15.45:0/2828327321 >> v1:172.21.15.45:6800/2598686815 conn(0x7f938c03a8c0 legacy=0x7f938c03cd80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:33.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.735+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9380005c50 con 0x7f93a0104bb0 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.735+0000 7f93977fe700 1 -- 172.21.15.45:0/2828327321 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f9390017010 con 0x7f93a0104bb0 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 >> v1:172.21.15.45:6800/2598686815 conn(0x7f938c03a8c0 legacy=0x7f938c03cd80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 >> v1:172.21.15.45:6789/0 conn(0x7f93a0104bb0 legacy=0x7f93a019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 shutdown_connections 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 >> 172.21.15.45:0/2828327321 conn(0x7f93a0100550 msgr2=0x7f93a01911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 shutdown_connections 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:32.738+0000 7f93a4f7d700 1 -- 172.21.15.45:0/2828327321 wait complete. 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-14T15:05:33.357 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-04-14T15:05:33.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2828327321' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-14T15:05:33.526 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.249+0000 7f4f61895200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:05:33.526 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.525+0000 7f4f61895200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:05:33.782 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.641+0000 7f4f61895200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:05:33.782 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.731+0000 7f4f61895200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:05:34.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:33.832+0000 7f4f61895200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:05:34.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:34 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:34.011+0000 7f4f61895200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:05:35.479 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.004+0000 7f4f61895200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:05:35.479 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.139+0000 7f4f61895200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:05:35.479 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.478+0000 7f4f61895200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:05:35.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.571+0000 7f4f61895200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:05:36.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.773+0000 7f4f61895200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:05:36.247 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:35.963+0000 7f4f61895200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:05:36.519 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:36.257+0000 7f4f61895200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:05:36.519 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:36.519+0000 7f4f61895200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:05:36.791 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:36.763+0000 7f4f61895200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:05:37.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:37.595+0000 7f4f61895200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:05:38.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:38.180+0000 7f4f61895200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:05:38.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:38.281+0000 7f4f61895200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:05:38.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:05:38.508+0000 7f4f61895200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: Active manager daemon y restarted 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: Activating manager daemon y 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: osdmap e3: 0 total, 0 up, 0 in 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: mgrmap e10: y(active, starting, since 0.00331278s) 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: Manager daemon y is now available 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:05:38.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:05:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:39 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:39] ENGINE Bus STARTING 2024-04-14T15:05:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:39 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:39] ENGINE Serving on http://172.21.15.45:8765 2024-04-14T15:05:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:39 smithi045 ceph-mon[29624]: mgrmap e11: y(active, since 1.00591s) 2024-04-14T15:05:40.078 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-14T15:05:40.078 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.781+0000 7fd8d7ed2700 1 Processor -- start 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.782+0000 7fd8d7ed2700 1 -- start start 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.782+0000 7fd8d7ed2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8d010ad30 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.782+0000 7fd8d5c6e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8d0106900 0x7fd8d0106d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34074/0 (socket says 172.21.15.45:34074) 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.782+0000 7fd8d5c6e700 1 -- 172.21.15.45:0/3499015504 learned_addr learned my addr 172.21.15.45:0/3499015504 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228133107 0 0) 0x7fd8d010ad30 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8bc0035b0 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3903131041 0 0) 0x7fd8bc0035b0 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d010bf10 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd8c0001580 con 0x7fd8d0106900 2024-04-14T15:05:40.079 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd8c0001c70 con 0x7fd8d0106900 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.783+0000 7fd8d4c6c700 1 -- 172.21.15.45:0/3499015504 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd8c0005960 con 0x7fd8d0106900 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/3499015504 >> v1:172.21.15.45:6789/0 conn(0x7fd8d0106900 legacy=0x7fd8d0106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/3499015504 shutdown_connections 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/3499015504 >> 172.21.15.45:0/3499015504 conn(0x7fd8d0101e60 msgr2=0x7fd8d01042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/3499015504 shutdown_connections 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/3499015504 wait complete. 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 Processor -- start 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.784+0000 7fd8d7ed2700 1 -- start start 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8d7ed2700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8d01a05f0 con 0x7fd8d0106900 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8d5c6e700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8d0106900 0x7fd8d007c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34088/0 (socket says 172.21.15.45:34088) 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8d5c6e700 1 -- 172.21.15.45:0/4291790329 learned_addr learned my addr 172.21.15.45:0/4291790329 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:40.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698363927 0 0) 0x7fd8d01a05f0 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8bc005580 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3338802946 0 0) 0x7fd8bc005580 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8d01a05f0 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd8c0001080 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1140318316 0 0) 0x7fd8d01a05f0 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d01a17d0 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d01a0820 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.785+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd8d01a0d60 con 0x7fd8d0106900 2024-04-14T15:05:40.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.786+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd8c0005660 con 0x7fd8d0106900 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.786+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fd8c0005fd0 con 0x7fd8d0106900 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.786+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 9) ==== 46536+0+0 (unknown 1243592525 0 0) 0x7fd8c0011780 con 0x7fd8d0106900 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.787+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6800/2598686815 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd8c0001080 con 0x7fd8bc03aa60 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.787+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(2..2 src has 1..2) ==== 897+0+0 (unknown 3402240117 0 0) 0x7fd8c0047fc0 con 0x7fd8d0106900 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.787+0000 7fd8d546d700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:33.988+0000 7fd8d546d700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:34.389+0000 7fd8d546d700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:35.190+0000 7fd8d546d700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:36.791+0000 7fd8d546d700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2598686815 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:38.511+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mgrmap(e 10) ==== 46338+0+0 (unknown 3016984278 0 0) 0x7fd8c0012d40 con 0x7fd8d0106900 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:38.511+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2598686815 conn(0x7fd8bc03aa60 legacy=0x7fd8bc03cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:05:40.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.513+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mgrmap(e 11) ==== 46430+0+0 (unknown 2199514505 0 0) 0x7fd8c0049460 con 0x7fd8d0106900 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.513+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6800/2191709230 -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd8c0004d90 con 0x7fd8bc03f410 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.514+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (unknown 0 0 4109177037) 0x7fd8c0004d90 con 0x7fd8bc03f410 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.521+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 --> v1:172.21.15.45:6800/2191709230 -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd8b4002630 con 0x7fd8bc03f410 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.521+0000 7fd8c6ffd700 1 -- 172.21.15.45:0/4291790329 <== mgr.14152 v1:172.21.15.45:6800/2191709230 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (unknown 0 0 3086460295) 0x7fd8b4002630 con 0x7fd8bc03f410 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6800/2191709230 conn(0x7fd8bc03f410 legacy=0x7fd8bc041800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 >> v1:172.21.15.45:6789/0 conn(0x7fd8d0106900 legacy=0x7fd8d007c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 shutdown_connections 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 >> 172.21.15.45:0/4291790329 conn(0x7fd8d0101e60 msgr2=0x7fd8d01911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 shutdown_connections 2024-04-14T15:05:40.083 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:39.522+0000 7fd8d7ed2700 1 -- 172.21.15.45:0/4291790329 wait complete. 2024-04-14T15:05:40.084 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-04-14T15:05:40.084 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-04-14T15:05:40.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:40 smithi045 ceph-mon[29624]: from='client.14156 v1:172.21.15.45:0/4291790329' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-14T15:05:40.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:40 smithi045 ceph-mon[29624]: from='client.14156 v1:172.21.15.45:0/4291790329' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-14T15:05:40.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:40 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:39] ENGINE Serving on https://172.21.15.45:7150 2024-04-14T15:05:40.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:40 smithi045 ceph-mon[29624]: [14/Apr/2024:15:05:39] ENGINE Bus STARTED 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.486+0000 7fdcfd268700 1 Processor -- start 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.486+0000 7fdcfd268700 1 -- start start 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.486+0000 7fdcfd268700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdcf8079e30 con 0x7fdcf81073a0 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdcf81073a0 0x7fdcf8109790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34166/0 (socket says 172.21.15.45:34166) 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf6ffd700 1 -- 172.21.15.45:0/3126730969 learned_addr learned my addr 172.21.15.45:0/3126730969 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650607402 0 0) 0x7fdcf8079e30 con 0x7fdcf81073a0 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdce40035b0 con 0x7fdcf81073a0 2024-04-14T15:05:41.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1490891767 0 0) 0x7fdce40035b0 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf810bec0 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fdce0001580 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fdce0004930 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.487+0000 7fdcf5ffb700 1 -- 172.21.15.45:0/3126730969 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fdce0005a70 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 -- 172.21.15.45:0/3126730969 >> v1:172.21.15.45:6789/0 conn(0x7fdcf81073a0 legacy=0x7fdcf8109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 -- 172.21.15.45:0/3126730969 shutdown_connections 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 -- 172.21.15.45:0/3126730969 >> 172.21.15.45:0/3126730969 conn(0x7fdcf8100dd0 msgr2=0x7fdcf81031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 -- 172.21.15.45:0/3126730969 shutdown_connections 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 -- 172.21.15.45:0/3126730969 wait complete. 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.488+0000 7fdcfd268700 1 Processor -- start 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdcfd268700 1 -- start start 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdcfd268700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fdcf81a05a0 con 0x7fdcf81073a0 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdcf6ffd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdcf81073a0 0x7fdcf8106c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34180/0 (socket says 172.21.15.45:34180) 2024-04-14T15:05:41.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdcf6ffd700 1 -- 172.21.15.45:0/2059858152 learned_addr learned my addr 172.21.15.45:0/2059858152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604349537 0 0) 0x7fdcf81a05a0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdce4005530 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3277445463 0 0) 0x7fdce4005530 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdcf81a05a0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.489+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fdce0001080 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 706045382 0 0) 0x7fdcf81a05a0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf81a1780 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf81a07d0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fdcf81a0d10 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fdce0004930 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.490+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fdce00060c0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.491+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 11) ==== 46430+0+0 (unknown 2199514505 0 0) 0x7fdce00117d0 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.491+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd8005120 con 0x7fdcf81073a0 2024-04-14T15:05:41.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.491+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7fdce0047ed0 con 0x7fdcf81073a0 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.496+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fdce0017730 con 0x7fdcf81073a0 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.563+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mgrmap(e 12) ==== 46536+0+0 (unknown 1940054160 0 0) 0x7fdce0001b60 con 0x7fdcf81073a0 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.713+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fdcd8000b90 con 0x7fdce403a900 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.814+0000 7fdceffff700 1 -- 172.21.15.45:0/2059858152 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (unknown 0 0 3317752739) 0x7fdcd8000b90 con 0x7fdce403a900 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 >> v1:172.21.15.45:6800/2191709230 conn(0x7fdce403a900 legacy=0x7fdce403cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 >> v1:172.21.15.45:6789/0 conn(0x7fdcf81073a0 legacy=0x7fdcf8106c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 shutdown_connections 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 >> 172.21.15.45:0/2059858152 conn(0x7fdcf8100dd0 msgr2=0x7fdcf81911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 shutdown_connections 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:40.816+0000 7fdcfd268700 1 -- 172.21.15.45:0/2059858152 wait complete. 2024-04-14T15:05:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-04-14T15:05:41.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:41 smithi045 ceph-mon[29624]: mgrmap e12: y(active, since 2s) 2024-04-14T15:05:41.812 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:41 smithi045 ceph-mon[29624]: from='client.14164 v1:172.21.15.45:0/2059858152' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:41.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:41 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:41.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:41 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:41.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:41 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:42.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qofPV1PsFvXGYX8qYA0jLeTOWlbGMOHfGyylTGAklwNc8ZdR8.O4C", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713107142, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-14T15:05:42.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.843+0000 7f74d0dfe700 1 Processor -- start 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.843+0000 7f74d0dfe700 1 -- start start 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.843+0000 7f74d0dfe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f74cc07a700 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74ca59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f74cc105130 0x7f74cc107560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37280/0 (socket says 172.21.15.45:37280) 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74ca59c700 1 -- 172.21.15.45:0/1010848132 learned_addr learned my addr 172.21.15.45:0/1010848132 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204801470 0 0) 0x7f74cc07a700 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74b40035b0 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1416759123 0 0) 0x7f74b40035b0 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc109c90 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f74c0001580 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f74c0004930 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.844+0000 7f74c959a700 1 -- 172.21.15.45:0/1010848132 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f74c0005a70 con 0x7f74cc105130 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.845+0000 7f74d0dfe700 1 -- 172.21.15.45:0/1010848132 >> v1:172.21.15.45:6789/0 conn(0x7f74cc105130 legacy=0x7f74cc107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.845+0000 7f74d0dfe700 1 -- 172.21.15.45:0/1010848132 shutdown_connections 2024-04-14T15:05:42.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.845+0000 7f74d0dfe700 1 -- 172.21.15.45:0/1010848132 >> 172.21.15.45:0/1010848132 conn(0x7f74cc100d50 msgr2=0x7f74cc103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.845+0000 7f74d0dfe700 1 -- 172.21.15.45:0/1010848132 shutdown_connections 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.845+0000 7f74d0dfe700 1 -- 172.21.15.45:0/1010848132 wait complete. 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.846+0000 7f74d0dfe700 1 Processor -- start 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.846+0000 7f74d0dfe700 1 -- start start 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.846+0000 7f74d0dfe700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f74cc1a1150 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.846+0000 7f74ca59c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f74cc105130 0x7f74cc1a0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37282/0 (socket says 172.21.15.45:37282) 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.846+0000 7f74ca59c700 1 -- 172.21.15.45:0/3731002039 learned_addr learned my addr 172.21.15.45:0/3731002039 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902000640 0 0) 0x7f74cc1a1150 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74b4005530 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2618309006 0 0) 0x7f74b4005530 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74cc1a1150 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f74c0001080 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1658045560 0 0) 0x7f74cc1a1150 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc1a1320 con 0x7f74cc105130 2024-04-14T15:05:42.907 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc1a1610 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f74cc1a5160 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f74c0005110 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.847+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f74c00063c0 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.848+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc064110 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.848+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 12) ==== 46536+0+0 (unknown 1940054160 0 0) 0x7f74c0011b50 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.849+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f74c0048180 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:41.858+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f74c0017800 con 0x7f74cc105130 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.093+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f74cc107d40 con 0x7f74b403aaa0 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.342+0000 7f74bb7fe700 1 -- 172.21.15.45:0/3731002039 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (unknown 0 0 2639463117) 0x7f74cc107d40 con 0x7f74b403aaa0 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 >> v1:172.21.15.45:6800/2191709230 conn(0x7f74b403aaa0 legacy=0x7f74b403cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 >> v1:172.21.15.45:6789/0 conn(0x7f74cc105130 legacy=0x7f74cc1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 shutdown_connections 2024-04-14T15:05:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 >> 172.21.15.45:0/3731002039 conn(0x7f74cc100d50 msgr2=0x7f74cc107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:42.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 shutdown_connections 2024-04-14T15:05:42.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:42.345+0000 7f74d0dfe700 1 -- 172.21.15.45:0/3731002039 wait complete. 2024-04-14T15:05:42.909 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-04-14T15:05:43.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:43 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:43.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:43 smithi045 ceph-mon[29624]: from='client.14166 v1:172.21.15.45:0/3731002039' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:43.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:43 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:44.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/974747248' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-14T15:05:44.911 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-04-14T15:05:44.911 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.331+0000 7f3c5168e700 1 Processor -- start 2024-04-14T15:05:44.911 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.331+0000 7f3c5168e700 1 -- start start 2024-04-14T15:05:44.911 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.331+0000 7f3c5168e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3c4c109c60 con 0x7f3c4c106e50 2024-04-14T15:05:44.911 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c4affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c4c106e50 0x7f3c4c107270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37286/0 (socket says 172.21.15.45:37286) 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c4affd700 1 -- 172.21.15.45:0/4285217091 learned_addr learned my addr 172.21.15.45:0/4285217091 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341681020 0 0) 0x7f3c4c109c60 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c380035b0 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2817071714 0 0) 0x7f3c380035b0 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c10ae40 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.332+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f3c44001900 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.333+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f3c44001ff0 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.333+0000 7f3c49ffb700 1 -- 172.21.15.45:0/4285217091 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f3c44005ce0 con 0x7f3c4c106e50 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.333+0000 7f3c5168e700 1 -- 172.21.15.45:0/4285217091 >> v1:172.21.15.45:6789/0 conn(0x7f3c4c106e50 legacy=0x7f3c4c107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.334+0000 7f3c5168e700 1 -- 172.21.15.45:0/4285217091 shutdown_connections 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.334+0000 7f3c5168e700 1 -- 172.21.15.45:0/4285217091 >> 172.21.15.45:0/4285217091 conn(0x7f3c4c100550 msgr2=0x7f3c4c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.334+0000 7f3c5168e700 1 -- 172.21.15.45:0/4285217091 shutdown_connections 2024-04-14T15:05:44.912 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.334+0000 7f3c5168e700 1 -- 172.21.15.45:0/4285217091 wait complete. 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.334+0000 7f3c5168e700 1 Processor -- start 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c5168e700 1 -- start start 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c5168e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3c4c199b90 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c4affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3c4c106e50 0x7f3c4c199480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37290/0 (socket says 172.21.15.45:37290) 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c4affd700 1 -- 172.21.15.45:0/974747248 learned_addr learned my addr 172.21.15.45:0/974747248 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278940060 0 0) 0x7f3c4c199b90 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c38005580 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4231662982 0 0) 0x7f3c38005580 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c4c199b90 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f3c44001400 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3329571086 0 0) 0x7f3c4c199b90 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.335+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c199d60 con 0x7f3c4c106e50 2024-04-14T15:05:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.336+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c199ff0 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.336+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3c4c104b60 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.336+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f3c44004de0 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.336+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f3c44006710 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.337+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 12) ==== 46536+0+0 (unknown 1940054160 0 0) 0x7f3c44011ea0 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.337+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c064110 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.337+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f3c44048450 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.343+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f3c44017b50 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.557+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3c4c10aa50 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.558+0000 7f3c43fff700 1 -- 172.21.15.45:0/974747248 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f3c44017450 con 0x7f3c4c106e50 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 >> v1:172.21.15.45:6800/2191709230 conn(0x7f3c3803cc70 legacy=0x7f3c3803f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 >> v1:172.21.15.45:6789/0 conn(0x7f3c4c106e50 legacy=0x7f3c4c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 shutdown_connections 2024-04-14T15:05:44.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 >> 172.21.15.45:0/974747248 conn(0x7f3c4c100550 msgr2=0x7f3c4c190160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:44.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 shutdown_connections 2024-04-14T15:05:44.915 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:43.560+0000 7f3c5168e700 1 -- 172.21.15.45:0/974747248 wait complete. 2024-04-14T15:05:44.919 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-14T15:05:44.919 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-14T15:05:44.927 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-14T15:05:44.927 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-14T15:05:44.928 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-14T15:05:44.928 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: Password: kw1vsvblq3 2024-04-14T15:05:44.930 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:44.931 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config directory 2024-04-14T15:05:44.931 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-04-14T15:05:46.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:46 smithi045 ceph-mon[29624]: mgrmap e13: y(active, since 6s) 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.357+0000 7f198fb25700 1 Processor -- start 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198fb25700 1 -- start start 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198fb25700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f198810ad10 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198d8c1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f19881068e0 0x7f1988106d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37296/0 (socket says 172.21.15.45:37296) 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198d8c1700 1 -- 172.21.15.45:0/726872492 learned_addr learned my addr 172.21.15.45:0/726872492 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1149167132 0 0) 0x7f198810ad10 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.358+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19740035b0 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030065671 0 0) 0x7f19740035b0 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198810bef0 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1984001580 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f1984004930 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198c8bf700 1 -- 172.21.15.45:0/726872492 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1984005960 con 0x7f19881068e0 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198fb25700 1 -- 172.21.15.45:0/726872492 >> v1:172.21.15.45:6789/0 conn(0x7f19881068e0 legacy=0x7f1988106d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:46.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198fb25700 1 -- 172.21.15.45:0/726872492 shutdown_connections 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198fb25700 1 -- 172.21.15.45:0/726872492 >> 172.21.15.45:0/726872492 conn(0x7f1988101e60 msgr2=0x7f19881042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198fb25700 1 -- 172.21.15.45:0/726872492 shutdown_connections 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.359+0000 7f198fb25700 1 -- 172.21.15.45:0/726872492 wait complete. 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f198fb25700 1 Processor -- start 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f198fb25700 1 -- start start 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f198fb25700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f198819ac20 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f198d8c1700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f19881068e0 0x7f198819a510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37308/0 (socket says 172.21.15.45:37308) 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f198d8c1700 1 -- 172.21.15.45:0/2853829482 learned_addr learned my addr 172.21.15.45:0/2853829482 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415161730 0 0) 0x7f198819ac20 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.360+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1974005530 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3930547238 0 0) 0x7f1974005530 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f198819ac20 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1984001080 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3541387194 0 0) 0x7f198819ac20 con 0x7f19881068e0 2024-04-14T15:05:46.158 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819adf0 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f198819b0e0 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f198819ec30 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f1984004930 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.361+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f1984005fc0 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.362+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f1984011760 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.362+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196c005150 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.362+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f19840480e0 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.367+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f1984017770 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.574+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f196c005440 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.574+0000 7f197affd700 1 -- 172.21.15.45:0/2853829482 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (unknown 689008156 0 0) 0x7f1984017070 con 0x7f19881068e0 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 >> v1:172.21.15.45:6800/2191709230 conn(0x7f197403ab10 legacy=0x7f197403cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 >> v1:172.21.15.45:6789/0 conn(0x7f19881068e0 legacy=0x7f198819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 shutdown_connections 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 >> 172.21.15.45:0/2853829482 conn(0x7f1988101e60 msgr2=0x7f19881042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:46.159 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 shutdown_connections 2024-04-14T15:05:46.160 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:45.576+0000 7f198fb25700 1 -- 172.21.15.45:0/2853829482 wait complete. 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.569+0000 7fa4c3ca0700 1 Processor -- start 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.569+0000 7fa4c3ca0700 1 -- start start 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c3ca0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa4bc10ad30 con 0x7fa4bc106900 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c1a3c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4bc106900 0x7fa4bc106d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37310/0 (socket says 172.21.15.45:37310) 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c1a3c700 1 -- 172.21.15.45:0/4204338244 learned_addr learned my addr 172.21.15.45:0/4204338244 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016336320 0 0) 0x7fa4bc10ad30 con 0x7fa4bc106900 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a80035b0 con 0x7fa4bc106900 2024-04-14T15:05:47.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1027736300 0 0) 0x7fa4a80035b0 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc10bf10 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fa4b0001580 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 <== mon.0 v1:172.21.15.45:6789/0 4 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fa4b0004930 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.570+0000 7fa4c0a3a700 1 -- 172.21.15.45:0/4204338244 <== mon.0 v1:172.21.15.45:6789/0 5 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fa4b0005960 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.571+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/4204338244 >> v1:172.21.15.45:6789/0 conn(0x7fa4bc106900 legacy=0x7fa4bc106d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.571+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/4204338244 shutdown_connections 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.571+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/4204338244 >> 172.21.15.45:0/4204338244 conn(0x7fa4bc101e60 msgr2=0x7fa4bc1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.571+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/4204338244 shutdown_connections 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.571+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/4204338244 wait complete. 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4c3ca0700 1 Processor -- start 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4c3ca0700 1 -- start start 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4c3ca0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa4bc19abd0 con 0x7fa4bc106900 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4c1a3c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4bc106900 0x7fa4bc19a4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37322/0 (socket says 172.21.15.45:37322) 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4c1a3c700 1 -- 172.21.15.45:0/874576378 learned_addr learned my addr 172.21.15.45:0/874576378 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:47.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333970594 0 0) 0x7fa4bc19abd0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a8005530 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 146229155 0 0) 0x7fa4a8005530 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa4bc19abd0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.572+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fa4b0001080 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1434161437 0 0) 0x7fa4bc19abd0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc19ada0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa4bc19b090 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa4bc19ebe0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fa4b0004930 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.573+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fa4b0006030 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.574+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7fa4b00117d0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.574+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a0005150 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.574+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7fa4b00481b0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.579+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fa4b001f3c0 con 0x7fa4bc106900 2024-04-14T15:05:47.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.902+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa4a0005440 con 0x7fa4bc106900 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.904+0000 7fa4aeffd700 1 -- 172.21.15.45:0/874576378 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (unknown 38603050 0 0) 0x7fa4b0014e70 con 0x7fa4bc106900 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.906+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 >> v1:172.21.15.45:6800/2191709230 conn(0x7fa4a803ab10 legacy=0x7fa4a803cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.906+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 >> v1:172.21.15.45:6789/0 conn(0x7fa4bc106900 legacy=0x7fa4bc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.906+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 shutdown_connections 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.906+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 >> 172.21.15.45:0/874576378 conn(0x7fa4bc101e60 msgr2=0x7fa4bc1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.907+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 shutdown_connections 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-14T15:05:46.907+0000 7fa4c3ca0700 1 -- 172.21.15.45:0/874576378 wait complete. 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /sbin/cephadm shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /sbin/cephadm shell 2024-04-14T15:05:47.461 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:05:47.462 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-04-14T15:05:47.497 INFO:tasks.cephadm:Fetching config... 2024-04-14T15:05:47.497 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:05:47.497 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-14T15:05:47.513 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-14T15:05:47.513 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:05:47.513 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-14T15:05:47.567 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-14T15:05:47.567 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:05:47.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-14T15:05:47.635 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-14T15:05:47.635 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:05:47.635 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-14T15:05:47.689 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-14T15:05:47.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXgSdPPFEAubTxmLRuP3NCCYlNy6O2QrkybXwEvH7TG5D6rTswr0pZTR4W5IitUuro60BqsAMuUy2lzLoMKiF9z2GNchM16VDu5Lvfs6M8qMHxtQ9N/Nhk7EE0cTFGgYpfkTT/3+mJfKg7956W5tmm560/lo+e9VFEWsvQVqYA0EZ3sNKe/SpS5XvyxVZa7KJaSh0wao7IkeTsVK7uigMUdb0UiiJ9qkZlCoYS5YiO+XZZ/VOEAzrBdZnSniSEaQUwrYlVk9Lqc4n69l5vj2odN7IVAZEfjYAJboL+xPnZt89JZ+VSBLFaO4GFNKJlg0Txb+JJjpa9byrONce510BChU9Ba8qcQd3GhWeQClKWVt6quASOsrn5ibUokkRB984FUBQ6EWvCWvy+Pr0dLvKtvmfsDTSsFF5tmMh+OGgk59MxEbPyQucdUJpURWis6sKdzqXNdkItokSL0065T4IK6dbQa6pV8HeYlzvm9OucYTzI9GIEgb6BJ2ZvPnpaYcM= ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-14T15:05:47.781 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXgSdPPFEAubTxmLRuP3NCCYlNy6O2QrkybXwEvH7TG5D6rTswr0pZTR4W5IitUuro60BqsAMuUy2lzLoMKiF9z2GNchM16VDu5Lvfs6M8qMHxtQ9N/Nhk7EE0cTFGgYpfkTT/3+mJfKg7956W5tmm560/lo+e9VFEWsvQVqYA0EZ3sNKe/SpS5XvyxVZa7KJaSh0wao7IkeTsVK7uigMUdb0UiiJ9qkZlCoYS5YiO+XZZ/VOEAzrBdZnSniSEaQUwrYlVk9Lqc4n69l5vj2odN7IVAZEfjYAJboL+xPnZt89JZ+VSBLFaO4GFNKJlg0Txb+JJjpa9byrONce510BChU9Ba8qcQd3GhWeQClKWVt6quASOsrn5ibUokkRB984FUBQ6EWvCWvy+Pr0dLvKtvmfsDTSsFF5tmMh+OGgk59MxEbPyQucdUJpURWis6sKdzqXNdkItokSL0065T4IK6dbQa6pV8HeYlzvm9OucYTzI9GIEgb6BJ2ZvPnpaYcM= ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:05:47.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXgSdPPFEAubTxmLRuP3NCCYlNy6O2QrkybXwEvH7TG5D6rTswr0pZTR4W5IitUuro60BqsAMuUy2lzLoMKiF9z2GNchM16VDu5Lvfs6M8qMHxtQ9N/Nhk7EE0cTFGgYpfkTT/3+mJfKg7956W5tmm560/lo+e9VFEWsvQVqYA0EZ3sNKe/SpS5XvyxVZa7KJaSh0wao7IkeTsVK7uigMUdb0UiiJ9qkZlCoYS5YiO+XZZ/VOEAzrBdZnSniSEaQUwrYlVk9Lqc4n69l5vj2odN7IVAZEfjYAJboL+xPnZt89JZ+VSBLFaO4GFNKJlg0Txb+JJjpa9byrONce510BChU9Ba8qcQd3GhWeQClKWVt6quASOsrn5ibUokkRB984FUBQ6EWvCWvy+Pr0dLvKtvmfsDTSsFF5tmMh+OGgk59MxEbPyQucdUJpURWis6sKdzqXNdkItokSL0065T4IK6dbQa6pV8HeYlzvm9OucYTzI9GIEgb6BJ2ZvPnpaYcM= ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-14T15:05:47.851 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXgSdPPFEAubTxmLRuP3NCCYlNy6O2QrkybXwEvH7TG5D6rTswr0pZTR4W5IitUuro60BqsAMuUy2lzLoMKiF9z2GNchM16VDu5Lvfs6M8qMHxtQ9N/Nhk7EE0cTFGgYpfkTT/3+mJfKg7956W5tmm560/lo+e9VFEWsvQVqYA0EZ3sNKe/SpS5XvyxVZa7KJaSh0wao7IkeTsVK7uigMUdb0UiiJ9qkZlCoYS5YiO+XZZ/VOEAzrBdZnSniSEaQUwrYlVk9Lqc4n69l5vj2odN7IVAZEfjYAJboL+xPnZt89JZ+VSBLFaO4GFNKJlg0Txb+JJjpa9byrONce510BChU9Ba8qcQd3GhWeQClKWVt6quASOsrn5ibUokkRB984FUBQ6EWvCWvy+Pr0dLvKtvmfsDTSsFF5tmMh+OGgk59MxEbPyQucdUJpURWis6sKdzqXNdkItokSL0065T4IK6dbQa6pV8HeYlzvm9OucYTzI9GIEgb6BJ2ZvPnpaYcM= ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:05:47.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-14T15:05:48.105 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:05:48.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/874576378' entity='client.admin' 2024-04-14T15:05:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.187+0000 7f0ef45af700 1 -- 172.21.15.45:0/101869129 >> v1:172.21.15.45:6789/0 conn(0x7f0eec101360 legacy=0x7f0eec101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.187+0000 7f0ef45af700 1 -- 172.21.15.45:0/101869129 shutdown_connections 2024-04-14T15:05:50.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.187+0000 7f0ef45af700 1 -- 172.21.15.45:0/101869129 >> 172.21.15.45:0/101869129 conn(0x7f0eec0fc900 msgr2=0x7f0eec0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.187+0000 7f0ef45af700 1 -- 172.21.15.45:0/101869129 shutdown_connections 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.187+0000 7f0ef45af700 1 -- 172.21.15.45:0/101869129 wait complete. 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ef45af700 1 Processor -- start 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ef45af700 1 -- start start 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ef45af700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0eec1997c0 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ef234b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0eec101360 0x7f0eec1990b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37340/0 (socket says 172.21.15.45:37340) 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ef234b700 1 -- 172.21.15.45:0/1655876777 learned_addr learned my addr 172.21.15.45:0/1655876777 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247374779 0 0) 0x7f0eec1997c0 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ed8005530 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2892167518 0 0) 0x7f0ed8005530 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0eec1997c0 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0ee8001480 con 0x7f0eec101360 2024-04-14T15:05:50.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3232768129 0 0) 0x7f0eec1997c0 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.188+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec199990 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.189+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0eec199c80 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.189+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0eec19d7d0 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.189+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0ee80036e0 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.189+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0ee8004d20 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.189+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f0ee8004fa0 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.190+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f0ee8001b60 con 0x7f0eec101360 2024-04-14T15:05:50.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.190+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed0005150 con 0x7f0eec101360 2024-04-14T15:05:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.195+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0ee8017710 con 0x7f0eec101360 2024-04-14T15:05:50.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.401+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 --> v1:172.21.15.45:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0ed0005440 con 0x7f0eec101360 2024-04-14T15:05:50.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.403+0000 7f0ee37fe700 1 -- 172.21.15.45:0/1655876777 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (unknown 3028693289 0 0) 0x7f0ee8017010 con 0x7f0eec101360 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 >> v1:172.21.15.45:6800/2191709230 conn(0x7f0ed803a930 legacy=0x7f0ed803cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 >> v1:172.21.15.45:6789/0 conn(0x7f0eec101360 legacy=0x7f0eec1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 shutdown_connections 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 >> 172.21.15.45:0/1655876777 conn(0x7f0eec0fc900 msgr2=0x7f0eec06aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 shutdown_connections 2024-04-14T15:05:50.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:50.405+0000 7f0ef45af700 1 -- 172.21.15.45:0/1655876777 wait complete. 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:50.734 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:51.044 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-14T15:05:51.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-14T15:05:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:05:51.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1655876777' entity='client.admin' 2024-04-14T15:05:51.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:51.662 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.745+0000 7faaced8e700 1 -- 172.21.15.45:0/2790908955 >> v1:172.21.15.45:6789/0 conn(0x7faac8102fb0 legacy=0x7faac81053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.745+0000 7faaced8e700 1 -- 172.21.15.45:0/2790908955 shutdown_connections 2024-04-14T15:05:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.745+0000 7faaced8e700 1 -- 172.21.15.45:0/2790908955 >> 172.21.15.45:0/2790908955 conn(0x7faac80fc900 msgr2=0x7faac80fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.746+0000 7faaced8e700 1 -- 172.21.15.45:0/2790908955 shutdown_connections 2024-04-14T15:05:52.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.746+0000 7faaced8e700 1 -- 172.21.15.45:0/2790908955 wait complete. 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.746+0000 7faaced8e700 1 Processor -- start 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.746+0000 7faaced8e700 1 -- start start 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faaced8e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7faac8197600 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faaccb2a700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faac8102fb0 0x7faac8196ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44014/0 (socket says 172.21.15.45:44014) 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faaccb2a700 1 -- 172.21.15.45:0/1296490156 learned_addr learned my addr 172.21.15.45:0/1296490156 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2856859696 0 0) 0x7faac8197600 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faab8005530 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3955530538 0 0) 0x7faab8005530 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faac8197600 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7faabc004750 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1559669646 0 0) 0x7faac8197600 con 0x7faac8102fb0 2024-04-14T15:05:52.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac81977d0 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faac8197ac0 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7faabc001b80 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7faabc0050a0 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.747+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7faac819b610 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.748+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7faabc005320 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.748+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac8064110 con 0x7faac8102fb0 2024-04-14T15:05:52.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.748+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7faabc0480b0 con 0x7faac8102fb0 2024-04-14T15:05:52.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.753+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7faabc017770 con 0x7faac8102fb0 2024-04-14T15:05:52.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.966+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7faac8105d50 con 0x7faab803a910 2024-04-14T15:05:52.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.969+0000 7faac5ffb700 1 -- 172.21.15.45:0/1296490156 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (unknown 0 0 0) 0x7faac8105d50 con 0x7faab803a910 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.971+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 >> v1:172.21.15.45:6800/2191709230 conn(0x7faab803a910 legacy=0x7faab803cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.971+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 >> v1:172.21.15.45:6789/0 conn(0x7faac8102fb0 legacy=0x7faac8196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.971+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 shutdown_connections 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.971+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 >> 172.21.15.45:0/1296490156 conn(0x7faac80fc900 msgr2=0x7faac81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.971+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 shutdown_connections 2024-04-14T15:05:52.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:52.972+0000 7faaced8e700 1 -- 172.21.15.45:0/1296490156 wait complete. 2024-04-14T15:05:53.736 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-04-14T15:05:53.736 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:05:53.736 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-04-14T15:05:53.756 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:05:53.756 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:05:53.813 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-04-14T15:05:53.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch host add smithi071 2024-04-14T15:05:54.054 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:05:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='client.14176 v1:172.21.15.45:0/1296490156' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:05:55.191 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:54 smithi045 ceph-mon[29624]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- 172.21.15.45:0/3473076707 >> v1:172.21.15.45:6789/0 conn(0x7f0874101540 legacy=0x7f0874101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- 172.21.15.45:0/3473076707 shutdown_connections 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- 172.21.15.45:0/3473076707 >> 172.21.15.45:0/3473076707 conn(0x7f08740fcdf0 msgr2=0x7f08740ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- 172.21.15.45:0/3473076707 shutdown_connections 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- 172.21.15.45:0/3473076707 wait complete. 2024-04-14T15:05:55.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 Processor -- start 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.387+0000 7f087b950700 1 -- start start 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f087b950700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0874198730 con 0x7f0874101540 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08796ec700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0874101540 0x7f0874198020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44046/0 (socket says 172.21.15.45:44046) 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08796ec700 1 -- 172.21.15.45:0/4056900414 learned_addr learned my addr 172.21.15.45:0/4056900414 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204720758 0 0) 0x7f0874198730 con 0x7f0874101540 2024-04-14T15:05:55.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f085c005530 con 0x7f0874101540 2024-04-14T15:05:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 655815772 0 0) 0x7f085c005530 con 0x7f0874101540 2024-04-14T15:05:55.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0874198730 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f086c004700 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2592058240 0 0) 0x7f0874198730 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0874198900 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0874198bf0 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.388+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f08741a0260 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.389+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f086c001ca0 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.389+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f086c004d40 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.389+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f086c004fc0 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.389+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0860005120 con 0x7f0874101540 2024-04-14T15:05:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.390+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f086c048010 con 0x7f0874101540 2024-04-14T15:05:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.394+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f086c0176d0 con 0x7f0874101540 2024-04-14T15:05:55.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:55.608+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) -- 0x7f0860000b90 con 0x7f085c03a950 2024-04-14T15:05:56.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:56 smithi045 ceph-mon[29624]: from='client.14178 v1:172.21.15.45:0/4056900414' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:05:56.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:56 smithi045 ceph-mon[29624]: Deploying cephadm binary to smithi071 2024-04-14T15:05:59.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.949+0000 7f08727fc700 1 -- 172.21.15.45:0/4056900414 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (unknown 0 0 1625968572) 0x7f0860000b90 con 0x7f085c03a950 2024-04-14T15:05:59.953 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-04-14T15:05:59.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:05:59 smithi045 ceph-mon[29624]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 >> v1:172.21.15.45:6800/2191709230 conn(0x7f085c03a950 legacy=0x7f085c03ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 >> v1:172.21.15.45:6789/0 conn(0x7f0874101540 legacy=0x7f0874198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 shutdown_connections 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 >> 172.21.15.45:0/4056900414 conn(0x7f08740fcdf0 msgr2=0x7f0874108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 shutdown_connections 2024-04-14T15:05:59.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:05:59.952+0000 7f087b950700 1 -- 172.21.15.45:0/4056900414 wait complete. 2024-04-14T15:06:00.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch host ls --format=json 2024-04-14T15:06:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:01.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:01.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:00 smithi045 ceph-mon[29624]: Added host smithi071 2024-04-14T15:06:01.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:01.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:02.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:01 smithi045 ceph-mon[29624]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:02.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:02.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.196+0000 7feed206d700 1 -- 172.21.15.45:0/146145444 >> v1:172.21.15.45:6789/0 conn(0x7feecc101540 legacy=0x7feecc101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:02.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.196+0000 7feed206d700 1 -- 172.21.15.45:0/146145444 shutdown_connections 2024-04-14T15:06:02.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.196+0000 7feed206d700 1 -- 172.21.15.45:0/146145444 >> 172.21.15.45:0/146145444 conn(0x7feecc076e40 msgr2=0x7feecc077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:02.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.196+0000 7feed206d700 1 -- 172.21.15.45:0/146145444 shutdown_connections 2024-04-14T15:06:02.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.196+0000 7feed206d700 1 -- 172.21.15.45:0/146145444 wait complete. 2024-04-14T15:06:02.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feed206d700 1 Processor -- start 2024-04-14T15:06:02.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feed206d700 1 -- start start 2024-04-14T15:06:02.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feed206d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7feecc100280 con 0x7feecc101540 2024-04-14T15:06:02.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feecb7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feecc101540 0x7feecc0fe390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38932/0 (socket says 172.21.15.45:38932) 2024-04-14T15:06:02.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feecb7fe700 1 -- 172.21.15.45:0/1026684018 learned_addr learned my addr 172.21.15.45:0/1026684018 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320408940 0 0) 0x7feecc100280 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.197+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8005530 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3265366663 0 0) 0x7feeb8005530 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feecc100280 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7feeb4004700 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 381044531 0 0) 0x7feecc100280 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc0feaa0 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feecc0fed90 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7feecc1a23e0 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7feeb4003760 con 0x7feecc101540 2024-04-14T15:06:02.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.198+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7feeb4004de0 con 0x7feecc101540 2024-04-14T15:06:02.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.199+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7feeb4004fa0 con 0x7feecc101540 2024-04-14T15:06:02.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.199+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7feeb4048f80 con 0x7feecc101540 2024-04-14T15:06:02.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.199+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeac005120 con 0x7feecc101540 2024-04-14T15:06:02.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.204+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7feeb40176f0 con 0x7feecc101540 2024-04-14T15:06:02.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.431+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feeac000b90 con 0x7feeb803a930 2024-04-14T15:06:02.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.433+0000 7feec8ff9700 1 -- 172.21.15.45:0/1026684018 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (unknown 0 0 1366000847) 0x7feeac000b90 con 0x7feeb803a930 2024-04-14T15:06:02.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:06:02.434 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 >> v1:172.21.15.45:6800/2191709230 conn(0x7feeb803a930 legacy=0x7feeb803cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 >> v1:172.21.15.45:6789/0 conn(0x7feecc101540 legacy=0x7feecc0fe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 shutdown_connections 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 >> 172.21.15.45:0/1026684018 conn(0x7feecc076e40 msgr2=0x7feecc06a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 shutdown_connections 2024-04-14T15:06:02.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:02.435+0000 7feed206d700 1 -- 172.21.15.45:0/1026684018 wait complete. 2024-04-14T15:06:03.041 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:02 smithi045 ceph-mon[29624]: from='client.14180 v1:172.21.15.45:0/1026684018' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:06:03.041 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:03.042 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-14T15:06:03.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd crush tunables default 2024-04-14T15:06:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:04.338 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:03 smithi045 ceph-mon[29624]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- 172.21.15.45:0/293815251 >> v1:172.21.15.45:6789/0 conn(0x7f0bc4108460 legacy=0x7f0bc4108840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- 172.21.15.45:0/293815251 shutdown_connections 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- 172.21.15.45:0/293815251 >> 172.21.15.45:0/293815251 conn(0x7f0bc40fdee0 msgr2=0x7f0bc4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- 172.21.15.45:0/293815251 shutdown_connections 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- 172.21.15.45:0/293815251 wait complete. 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 Processor -- start 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.661+0000 7f0bcbd21700 1 -- start start 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bcbd21700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0bc4076ca0 con 0x7f0bc4108460 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc9abd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0bc4108460 0x7f0bc4074db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38950/0 (socket says 172.21.15.45:38950) 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc9abd700 1 -- 172.21.15.45:0/1282523747 learned_addr learned my addr 172.21.15.45:0/1282523747 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345447452 0 0) 0x7f0bc4076ca0 con 0x7f0bc4108460 2024-04-14T15:06:04.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bac005530 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2838464599 0 0) 0x7f0bac005530 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0bc4076ca0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0bbc0047e0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3601851471 0 0) 0x7f0bc4076ca0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc40754c0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc40757b0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.662+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0bc41a25f0 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.663+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0bbc001a80 con 0x7f0bc4108460 2024-04-14T15:06:04.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.663+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f0bbc004dc0 con 0x7f0bc4108460 2024-04-14T15:06:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.663+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f0bbc004f80 con 0x7f0bc4108460 2024-04-14T15:06:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.663+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(3..3 src has 1..3) ==== 1069+0+0 (unknown 3188217654 0 0) 0x7f0bbc048e00 con 0x7f0bc4108460 2024-04-14T15:06:04.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.664+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0005120 con 0x7f0bc4108460 2024-04-14T15:06:04.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.668+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0bc4108fe0 con 0x7f0bc4108460 2024-04-14T15:06:04.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.879+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0bb0005410 con 0x7f0bc4108460 2024-04-14T15:06:05.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:04.998+0000 7f0bc2ffd700 1 -- 172.21.15.45:0/1282523747 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (unknown 3126668360 0 0) 0x7f0bbc017630 con 0x7f0bc4108460 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 >> v1:172.21.15.45:6800/2191709230 conn(0x7f0bac03a9d0 legacy=0x7f0bac03ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 >> v1:172.21.15.45:6789/0 conn(0x7f0bc4108460 legacy=0x7f0bc4074db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 shutdown_connections 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 >> 172.21.15.45:0/1282523747 conn(0x7f0bc40fdee0 msgr2=0x7f0bc410a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 shutdown_connections 2024-04-14T15:06:05.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:05.001+0000 7f0bcbd21700 1 -- 172.21.15.45:0/1282523747 wait complete. 2024-04-14T15:06:05.003 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-04-14T15:06:05.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-14T15:06:05.518 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-04-14T15:06:05.518 INFO:tasks.cephadm:Adding mon.c on smithi045 2024-04-14T15:06:05.519 INFO:tasks.cephadm:Adding mon.b on smithi071 2024-04-14T15:06:05.519 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply mon '3;smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b' 2024-04-14T15:06:05.825 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-14T15:06:05.873 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.446+0000 7fee27f6a700 1 -- 172.21.15.71:0/1409410486 >> v1:172.21.15.45:6789/0 conn(0x7fee20101540 legacy=0x7fee20101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.446+0000 7fee27f6a700 1 -- 172.21.15.71:0/1409410486 shutdown_connections 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.446+0000 7fee27f6a700 1 -- 172.21.15.71:0/1409410486 >> 172.21.15.71:0/1409410486 conn(0x7fee20076e40 msgr2=0x7fee20077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.447+0000 7fee27f6a700 1 -- 172.21.15.71:0/1409410486 shutdown_connections 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.447+0000 7fee27f6a700 1 -- 172.21.15.71:0/1409410486 wait complete. 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.447+0000 7fee27f6a700 1 Processor -- start 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.447+0000 7fee27f6a700 1 -- start start 2024-04-14T15:06:07.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee27f6a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fee20198760 con 0x7fee20101540 2024-04-14T15:06:07.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee25d06700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee20101540 0x7fee20198050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.71:40550/0 (socket says 172.21.15.71:40550) 2024-04-14T15:06:07.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee25d06700 1 -- 172.21.15.71:0/1242620952 learned_addr learned my addr 172.21.15.71:0/1242620952 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:06:07.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300167235 0 0) 0x7fee20198760 con 0x7fee20101540 2024-04-14T15:06:07.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee0c005580 con 0x7fee20101540 2024-04-14T15:06:07.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 772533455 0 0) 0x7fee0c005580 con 0x7fee20101540 2024-04-14T15:06:07.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fee20198760 con 0x7fee20101540 2024-04-14T15:06:07.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee1c001840 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.448+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1135718306 0 0) 0x7fee20198760 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee20198930 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fee20198c20 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fee201a0290 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fee1c001ff0 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7fee1c0050c0 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee04005120 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.449+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7fee1c005280 con 0x7fee20101540 2024-04-14T15:06:07.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.450+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7fee1c0483a0 con 0x7fee20101540 2024-04-14T15:06:07.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.455+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fee1c014fa0 con 0x7fee20101540 2024-04-14T15:06:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:06:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:07.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.663+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b", "target": ["mon-mgr", ""]}) -- 0x7fee04000c60 con 0x7fee0c03a950 2024-04-14T15:06:07.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.669+0000 7fee16ffd700 1 -- 172.21.15.71:0/1242620952 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 3265049985) 0x7fee04000c60 con 0x7fee0c03a950 2024-04-14T15:06:07.670 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mon update... 2024-04-14T15:06:07.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 >> v1:172.21.15.45:6800/2191709230 conn(0x7fee0c03a950 legacy=0x7fee0c03ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:07.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 >> v1:172.21.15.45:6789/0 conn(0x7fee20101540 legacy=0x7fee20198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:07.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 shutdown_connections 2024-04-14T15:06:07.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 >> 172.21.15.71:0/1242620952 conn(0x7fee20076e40 msgr2=0x7fee20108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:07.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 shutdown_connections 2024-04-14T15:06:07.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:07.671+0000 7fee27f6a700 1 -- 172.21.15.71:0/1242620952 wait complete. 2024-04-14T15:06:08.334 DEBUG:teuthology.orchestra.run.smithi045:mon.c> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.c.service 2024-04-14T15:06:08.337 DEBUG:teuthology.orchestra.run.smithi071:mon.b> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.b.service 2024-04-14T15:06:08.339 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-14T15:06:08.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph mon dump -f json 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:08.364 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:09.255 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:09.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:09 smithi045 ceph-mon[29624]: from='client.14184 v1:172.21.15.71:0/1242620952' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:09 smithi045 ceph-mon[29624]: Saving service mon spec with placement smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b;count:3 2024-04-14T15:06:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:09 smithi045 ceph-mon[29624]: Deploying daemon mon.b on smithi071 2024-04-14T15:06:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:10 smithi045 ceph-mon[29624]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:11 smithi071 systemd[1]: Starting Ceph mon.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 -- 172.21.15.71:0/610422837 >> v1:172.21.15.45:6789/0 conn(0x7f90001062c0 legacy=0x7f90001066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 -- 172.21.15.71:0/610422837 shutdown_connections 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 -- 172.21.15.71:0/610422837 >> 172.21.15.71:0/610422837 conn(0x7f90000fc520 msgr2=0x7f90000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 -- 172.21.15.71:0/610422837 shutdown_connections 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 -- 172.21.15.71:0/610422837 wait complete. 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.047+0000 7f9005af3700 1 Processor -- start 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.048+0000 7f9005af3700 1 -- start start 2024-04-14T15:06:12.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.048+0000 7f9005af3700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9000198880 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.048+0000 7f8fff7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f90001062c0 0x7f9000198170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.71:40566/0 (socket says 172.21.15.71:40566) 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.048+0000 7f8fff7fe700 1 -- 172.21.15.71:0/1440931310 learned_addr learned my addr 172.21.15.71:0/1440931310 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077017380 0 0) 0x7f9000198880 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fe8005580 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2994063546 0 0) 0x7f8fe8005580 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9000198880 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8ff0001370 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3996555453 0 0) 0x7f9000198880 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9000198a50 con 0x7f90001062c0 2024-04-14T15:06:12.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9000198d40 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f90001a03b0 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f8ff0001190 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.049+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 170+0+0 (unknown 3535813903 0 0) 0x7f8ff0004cf0 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.050+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f8ff0004f30 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.050+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec005120 con 0x7f90001062c0 2024-04-14T15:06:12.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.050+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7f8ff0048e10 con 0x7f90001062c0 2024-04-14T15:06:12.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.056+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f8ff0017640 con 0x7f90001062c0 2024-04-14T15:06:12.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:12 smithi045 ceph-mon[29624]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.356+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8fec005410 con 0x7f90001062c0 2024-04-14T15:06:12.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.357+0000 7f8ffcff9700 1 -- 172.21.15.71:0/1440931310 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+694 (unknown 2237029548 0 2893813925) 0x7f8ff0016f40 con 0x7f90001062c0 2024-04-14T15:06:12.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:06:12.359 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","modified":"2024-04-14T15:04:37.102829Z","created":"2024-04-14T15:04:37.102829Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 >> v1:172.21.15.45:6800/2191709230 conn(0x7f8fe803a950 legacy=0x7f8fe803ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 >> v1:172.21.15.45:6789/0 conn(0x7f90001062c0 legacy=0x7f9000198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 shutdown_connections 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 >> 172.21.15.71:0/1440931310 conn(0x7f90000fc520 msgr2=0x7f90000fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 shutdown_connections 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:12.359+0000 7f9005af3700 1 -- 172.21.15.71:0/1440931310 wait complete. 2024-04-14T15:06:12.361 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 podman[33118]: 2024-04-14 15:06:12.083404295 +0000 UTC m=+0.025924974 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 podman[33118]: 2024-04-14 15:06:12.22483322 +0000 UTC m=+0.167353884 container create fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=main-a971fd5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 podman[33118]: 2024-04-14 15:06:12.419263988 +0000 UTC m=+0.361784651 container init fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 podman[33118]: 2024-04-14 15:06:12.42248343 +0000 UTC m=+0.365004098 container start fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b, RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: set uid:gid to 167:167 (ceph:ceph) 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev), process ceph-mon, pid 2 2024-04-14T15:06:12.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pidfile_write: ignore empty --pid-file 2024-04-14T15:06:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: load: jerasure load: lrc 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: RocksDB version: 7.9.2 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Git sha 0 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Compile date 2024-04-06 20:28:24 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: DB SUMMARY 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: DB Session ID: AH87I33XL684GXXP6CN7 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: CURRENT file: CURRENT 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: IDENTITY file: IDENTITY 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.error_if_exists: 0 2024-04-14T15:06:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.create_if_missing: 0 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.paranoid_checks: 1 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.env: 0x564236e18c60 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.fs: PosixFileSystem 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.info_log: 0x564239297c60 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_file_opening_threads: 16 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.statistics: (nil) 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.use_fsync: 0 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_log_file_size: 0 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.keep_log_file_num: 1000 2024-04-14T15:06:12.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.recycle_log_file_num: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_fallocate: 1 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_mmap_reads: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_mmap_writes: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.use_direct_reads: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.create_missing_column_families: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.db_log_dir: 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.wal_dir: 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-14T15:06:12.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.advise_random_on_open: 1 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.db_write_buffer_size: 0 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.write_buffer_manager: 0x5642392681e0 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.rate_limiter: (nil) 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-14T15:06:12.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.wal_recovery_mode: 2 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enable_thread_tracking: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enable_pipelined_write: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.unordered_write: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.row_cache: None 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.wal_filter: None 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_ingest_behind: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.two_write_queues: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.manual_wal_flush: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.wal_compression: 0 2024-04-14T15:06:12.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.atomic_flush: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.log_readahead_size: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.best_efforts_recovery: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.allow_data_in_errors: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.db_host_id: __hostname__ 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_background_jobs: 2 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_background_compactions: -1 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_subcompactions: 1 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-14T15:06:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_total_wal_size: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_open_files: -1 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bytes_per_sync: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_readahead_size: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_background_flushes: -1 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Compression algorithms supported: 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kZSTD supported: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kXpressCompression supported: 0 2024-04-14T15:06:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kLZ4HCCompression supported: 1 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kZlibCompression supported: 1 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kSnappyCompression supported: 1 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kLZ4Compression supported: 1 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: kBZip2Compression supported: 0 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.merge_operator: 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_filter: None 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_filter_factory: None 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.sst_partitioner_factory: None 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-14T15:06:12.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564239297e20) 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_type: 0 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: data_block_index_type: 0 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_shortening: 1 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: checksum: 4 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: no_block_cache: 0 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache: 0x56423929b090 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_options: 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: capacity : 536870912 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: num_shard_bits : 4 2024-04-14T15:06:12.867 INFO:journalctl@ceph.mon.b.smithi071.stdout: strict_capacity_limit : 0 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_cache_compressed: (nil) 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: persistent_cache: (nil) 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_size: 4096 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_size_deviation: 10 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_restart_interval: 16 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: index_block_restart_interval: 1 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: metadata_block_size: 4096 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: partition_filters: 0 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: use_delta_encoding: 1 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: filter_policy: bloomfilter 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: whole_key_filtering: 1 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: verify_compression: 0 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-04-14T15:06:12.868 INFO:journalctl@ceph.mon.b.smithi071.stdout: format_version: 5 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_index_compression: 1 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: block_align: 0 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: max_auto_readahead_size: 262144 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: prepopulate_block_cache: 0 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: initial_auto_readahead_size: 8192 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.write_buffer_size: 33554432 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_write_buffer_number: 2 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression: NoCompression 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression: Disabled 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.prefix_extractor: nullptr 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.num_levels: 7 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-14T15:06:12.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.level: 32767 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.strategy: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:06:12.870 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.enabled: false 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.target_file_size_base: 67108864 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-14T15:06:12.871 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.arena_block_size: 1048576 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.disable_auto_compactions: 0 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-14T15:06:12.872 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.inplace_update_support: 0 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-14T15:06:12.873 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.bloom_locality: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.max_successive_merges: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.paranoid_file_checks: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.force_consistency_checks: 1 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.report_bg_io_stats: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.ttl: 2592000 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enable_blob_files: false 2024-04-14T15:06:12.874 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.min_blob_size: 0 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_file_size: 268435456 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.blob_file_starting_level: 0 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e4ae9989-bb43-41dc-96ca-6049031ff780 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107172452344, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107172452773, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1613, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713107172, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e4ae9989-bb43-41dc-96ca-6049031ff780", "db_session_id": "AH87I33XL684GXXP6CN7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107172452851, "job": 1, "event": "recovery_finished"} 2024-04-14T15:06:12.875 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56423938a000 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: DB pointer 0x564239374000 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** DB Stats ** 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:06:12.876 INFO:journalctl@ceph.mon.b.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-14T15:06:12.877 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache BinnedLRUCache@0x56423929b090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: using public_addrv v1:172.21.15.71:6789/0 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: starting mon.b rank -1 at public addrs v1:172.21.15.71:6789/0 at bind addrs v1:172.21.15.71:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(???) e0 preinit fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).mds e1 new map 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-14T15:06:12.878 INFO:journalctl@ceph.mon.b.smithi071.stdout: e1 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout: legacy client fscid: -1 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout: No filesystems configured 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:12.879 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:12.880 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:12.881 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: from='client.14184 v1:172.21.15.71:0/1242620952' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Saving service mon spec with placement smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b;count:3 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: Deploying daemon mon.b on smithi071 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 ceph-mon[33150]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 bash[33118]: fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 2024-04-14T15:06:12.882 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:12 smithi071 systemd[1]: Started Ceph mon.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:06:14.050 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-14T15:06:14.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph mon dump -f json 2024-04-14T15:06:14.295 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.487+0000 7f6be2055700 1 -- 172.21.15.71:0/1184688792 >> v1:172.21.15.45:6789/0 conn(0x7f6bdc100df0 legacy=0x7f6bdc1011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.487+0000 7f6be2055700 1 -- 172.21.15.71:0/1184688792 shutdown_connections 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.487+0000 7f6be2055700 1 -- 172.21.15.71:0/1184688792 >> 172.21.15.71:0/1184688792 conn(0x7f6bdc077ba0 msgr2=0x7f6bdc077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.487+0000 7f6be2055700 1 -- 172.21.15.71:0/1184688792 shutdown_connections 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.487+0000 7f6be2055700 1 -- 172.21.15.71:0/1184688792 wait complete. 2024-04-14T15:06:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.488+0000 7f6be2055700 1 Processor -- start 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.488+0000 7f6be2055700 1 -- start start 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.488+0000 7f6be2055700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6bdc1a36a0 con 0x7f6bdc197b70 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.489+0000 7f6be2055700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6bdc1a4880 con 0x7f6bdc100df0 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.489+0000 7f6bdb7fe700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f6bdc100df0 0x7f6bdc197460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:57772/0 (socket says 172.21.15.71:57772) 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.489+0000 7f6bdb7fe700 1 -- 172.21.15.71:0/3392072406 learned_addr learned my addr 172.21.15.71:0/3392072406 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.489+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201774934 0 0) 0x7f6bdc1a36a0 con 0x7f6bdc197b70 2024-04-14T15:06:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.489+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc40046c0 con 0x7f6bdc197b70 2024-04-14T15:06:17.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2560270404 0 0) 0x7f6bc40046c0 con 0x7f6bdc197b70 2024-04-14T15:06:17.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6bdc1a36a0 con 0x7f6bdc197b70 2024-04-14T15:06:17.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 1569335970 0 0) 0x7f6bd00017e0 con 0x7f6bdc197b70 2024-04-14T15:06:17.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4145787773 0 0) 0x7f6bdc1a36a0 con 0x7f6bdc197b70 2024-04-14T15:06:17.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 >> v1:172.21.15.71:6789/0 conn(0x7f6bdc100df0 legacy=0x7f6bdc197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc1a5a60 con 0x7f6bdc197b70 2024-04-14T15:06:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc1a3870 con 0x7f6bdc197b70 2024-04-14T15:06:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.490+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a3d70 con 0x7f6bdc197b70 2024-04-14T15:06:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.492+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bdc064110 con 0x7f6bdc197b70 2024-04-14T15:06:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.492+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6bd0003b10 con 0x7f6bdc197b70 2024-04-14T15:06:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.492+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 239+0+0 (unknown 1569335970 0 0) 0x7f6bd00049f0 con 0x7f6bdc197b70 2024-04-14T15:06:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.493+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f6bd0010190 con 0x7f6bdc197b70 2024-04-14T15:06:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.493+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7f6bd00479e0 con 0x7f6bdc197b70 2024-04-14T15:06:17.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.502+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f6bd0016150 con 0x7f6bdc197b70 2024-04-14T15:06:17.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.781+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6bdc10b230 con 0x7f6bdc197b70 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: mon.a calling monitor election 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: mon.b calling monitor election 2024-04-14T15:06:17.783 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: monmap epoch 2 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: last_changed 2024-04-14T15:06:12.472036+0000 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: min_mon_release 19 (squid) 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: election_strategy: 1 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: fsmap 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:17.784 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: mgrmap e13: y(active, since 38s) 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: overall HEALTH_OK 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:17.785 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:17.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.782+0000 7f6bd8ff9700 1 -- 172.21.15.71:0/3392072406 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+909 (unknown 3557084514 0 2263136297) 0x7f6bd0015a50 con 0x7f6bdc197b70 2024-04-14T15:06:17.785 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:06:17.785 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","modified":"2024-04-14T15:06:12.472036Z","created":"2024-04-14T15:04:37.102829Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 >> v1:172.21.15.45:6800/2191709230 conn(0x7f6bc403aaa0 legacy=0x7f6bc403cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 >> v1:172.21.15.45:6789/0 conn(0x7f6bdc197b70 legacy=0x7f6bdc1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 shutdown_connections 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 >> 172.21.15.71:0/3392072406 conn(0x7f6bdc077ba0 msgr2=0x7f6bdc1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 shutdown_connections 2024-04-14T15:06:17.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:17.785+0000 7f6be2055700 1 -- 172.21.15.71:0/3392072406 wait complete. 2024-04-14T15:06:17.787 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 2 2024-04-14T15:06:17.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: mon.a calling monitor election 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: mon.b calling monitor election 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-14T15:06:17.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: monmap epoch 2 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: last_changed 2024-04-14T15:06:12.472036+0000 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: min_mon_release 19 (squid) 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: election_strategy: 1 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: fsmap 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: mgrmap e13: y(active, since 38s) 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: overall HEALTH_OK 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:17.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:17.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:18 smithi045 ceph-mon[29624]: Deploying daemon mon.c on smithi045 2024-04-14T15:06:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/3392072406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:18 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:18 smithi071 ceph-mon[33150]: Deploying daemon mon.c on smithi045 2024-04-14T15:06:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/3392072406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:18 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:19.384 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-14T15:06:19.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph mon dump -f json 2024-04-14T15:06:19.624 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:06:19.743 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:19 smithi071 ceph-mon[33150]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:19.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:19 smithi045 ceph-mon[29624]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:20.556 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 systemd[1]: Starting Ceph mon.c for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:06:20.809 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 podman[37768]: 2024-04-14 15:06:20.597139306 +0000 UTC m=+0.018832818 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:06:20.810 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 podman[37768]: 2024-04-14 15:06:20.698738979 +0000 UTC m=+0.120432523 container create 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:06:20.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.925+0000 7faddef60700 1 -- 172.21.15.71:0/3885880409 >> v1:172.21.15.45:6789/0 conn(0x7fadd81074a0 legacy=0x7fadd8107880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:20.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.925+0000 7faddef60700 1 -- 172.21.15.71:0/3885880409 shutdown_connections 2024-04-14T15:06:20.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.925+0000 7faddef60700 1 -- 172.21.15.71:0/3885880409 >> 172.21.15.71:0/3885880409 conn(0x7fadd80fcf00 msgr2=0x7fadd80ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:20.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.925+0000 7faddef60700 1 -- 172.21.15.71:0/3885880409 shutdown_connections 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.925+0000 7faddef60700 1 -- 172.21.15.71:0/3885880409 wait complete. 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddef60700 1 Processor -- start 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddef60700 1 -- start start 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddef60700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fadd81a1610 con 0x7fadd8195a90 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddef60700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fadd81a27f0 con 0x7fadd81074a0 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddccfc700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fadd81074a0 0x7fadd8195380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:57784/0 (socket says 172.21.15.71:57784) 2024-04-14T15:06:20.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7faddccfc700 1 -- 172.21.15.71:0/754766974 learned_addr learned my addr 172.21.15.71:0/754766974 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.926+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467733375 0 0) 0x7fadd81a27f0 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadc80035b0 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3562588946 0 0) 0x7fadd81a1610 con 0x7fadd8195a90 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadd81a27f0 con 0x7fadd8195a90 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3342087643 0 0) 0x7fadc80035b0 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadd81a1610 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 1569335970 0 0) 0x7fadcc004830 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1846121327 0 0) 0x7fadd81a27f0 con 0x7fadd8195a90 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadc80035b0 con 0x7fadd8195a90 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 239+0+0 (unknown 1569335970 0 0) 0x7fadc40017e0 con 0x7fadd8195a90 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4250869039 0 0) 0x7fadd81a1610 con 0x7fadd81074a0 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 >> v1:172.21.15.45:6789/0 conn(0x7fadd8195a90 legacy=0x7fadd819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:20.929 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd81a39d0 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fadd81a17e0 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fadcc003b10 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 239+0+0 (unknown 1569335970 0 0) 0x7fadcc006110 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.927+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fadd81a1ce0 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.928+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7fadcc011a20 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.928+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7fadcc047fd0 con 0x7fadd81074a0 2024-04-14T15:06:20.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.928+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd8064110 con 0x7fadd81074a0 2024-04-14T15:06:20.933 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.933+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fadd8108020 con 0x7fadd81074a0 2024-04-14T15:06:20.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:20.961+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fadcc016940 con 0x7fadd81074a0 2024-04-14T15:06:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:21.217+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fadd8196800 con 0x7fadd81074a0 2024-04-14T15:06:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 podman[37768]: 2024-04-14 15:06:20.901613112 +0000 UTC m=+0.323306625 container init 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 podman[37768]: 2024-04-14 15:06:20.905046682 +0000 UTC m=+0.326740189 container start 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=main-a971fd5) 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: set uid:gid to 167:167 (ceph:ceph) 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: ceph version 19.0.0-2778-ga971fd5f (a971fd5f5719038c70b817bafacd6d5400b1d888) squid (dev), process ceph-mon, pid 2 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pidfile_write: ignore empty --pid-file 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: load: jerasure load: lrc 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: RocksDB version: 7.9.2 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Git sha 0 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Compile date 2024-04-06 20:28:24 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: DB SUMMARY 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: DB Session ID: G48VAIIGQUDYCYKO0PO1 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: CURRENT file: CURRENT 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: IDENTITY file: IDENTITY 2024-04-14T15:06:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.error_if_exists: 0 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.create_if_missing: 0 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.paranoid_checks: 1 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.env: 0x564b77197c60 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.fs: PosixFileSystem 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.info_log: 0x564b7931bc20 2024-04-14T15:06:21.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_file_opening_threads: 16 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.statistics: (nil) 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.use_fsync: 0 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_log_file_size: 0 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.keep_log_file_num: 1000 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.recycle_log_file_num: 0 2024-04-14T15:06:21.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_fallocate: 1 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_mmap_reads: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_mmap_writes: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.use_direct_reads: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.create_missing_column_families: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.db_log_dir: 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.wal_dir: 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.advise_random_on_open: 1 2024-04-14T15:06:21.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.db_write_buffer_size: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.write_buffer_manager: 0x564b792ec1e0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.rate_limiter: (nil) 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.wal_recovery_mode: 2 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enable_thread_tracking: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enable_pipelined_write: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.unordered_write: 0 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-14T15:06:21.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.row_cache: None 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.wal_filter: None 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_ingest_behind: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.two_write_queues: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.manual_wal_flush: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.wal_compression: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.atomic_flush: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.log_readahead_size: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.best_efforts_recovery: 0 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-14T15:06:21.253 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.allow_data_in_errors: 0 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.db_host_id: __hostname__ 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_background_jobs: 2 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_background_compactions: -1 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_subcompactions: 1 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_total_wal_size: 0 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_open_files: -1 2024-04-14T15:06:21.254 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bytes_per_sync: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_readahead_size: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_background_flushes: -1 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Compression algorithms supported: 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kZSTD supported: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kXpressCompression supported: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kLZ4HCCompression supported: 1 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kZlibCompression supported: 1 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kSnappyCompression supported: 1 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kLZ4Compression supported: 1 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: kBZip2Compression supported: 0 2024-04-14T15:06:21.255 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.merge_operator: 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_filter: None 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_filter_factory: None 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.sst_partitioner_factory: None 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564b7931bde0) 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-14T15:06:21.256 INFO:journalctl@ceph.mon.c.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: index_type: 0 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: data_block_index_type: 0 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: index_shortening: 1 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: checksum: 4 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: no_block_cache: 0 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_cache: 0x564b7931f090 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_cache_options: 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: capacity : 536870912 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: num_shard_bits : 4 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: strict_capacity_limit : 0 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-04-14T15:06:21.257 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_cache_compressed: (nil) 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: persistent_cache: (nil) 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_size: 4096 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_size_deviation: 10 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_restart_interval: 16 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: index_block_restart_interval: 1 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: metadata_block_size: 4096 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: partition_filters: 0 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: use_delta_encoding: 1 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: filter_policy: bloomfilter 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: whole_key_filtering: 1 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: verify_compression: 0 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: format_version: 5 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: enable_index_compression: 1 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: block_align: 0 2024-04-14T15:06:21.258 INFO:journalctl@ceph.mon.c.smithi045.stdout: max_auto_readahead_size: 262144 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout: prepopulate_block_cache: 0 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout: initial_auto_readahead_size: 8192 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.write_buffer_size: 33554432 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_write_buffer_number: 2 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression: NoCompression 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression: Disabled 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.prefix_extractor: nullptr 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.num_levels: 7 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-14T15:06:21.259 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.level: 32767 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.strategy: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.enabled: false 2024-04-14T15:06:21.260 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.target_file_size_base: 67108864 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-14T15:06:21.261 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.arena_block_size: 1048576 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.disable_auto_compactions: 0 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-14T15:06:21.262 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.inplace_update_support: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.bloom_locality: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.max_successive_merges: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.paranoid_file_checks: 0 2024-04-14T15:06:21.263 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.force_consistency_checks: 1 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.report_bg_io_stats: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.ttl: 2592000 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enable_blob_files: false 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.min_blob_size: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_file_size: 268435456 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.blob_file_starting_level: 0 2024-04-14T15:06:21.264 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4dd29e85-946d-4df2-a643-70dc39b4930d 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107180934969, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107180935356, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1703, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713107180, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4dd29e85-946d-4df2-a643-70dc39b4930d", "db_session_id": "G48VAIIGQUDYCYKO0PO1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713107180935439, "job": 1, "event": "recovery_finished"} 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564b7940e000 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: DB pointer 0x564b793f8000 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout: ** DB Stats ** 2024-04-14T15:06:21.265 INFO:journalctl@ceph.mon.c.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:21.266 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:06:21.267 INFO:journalctl@ceph.mon.c.smithi045.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: Block cache BinnedLRUCache@0x564b7931f090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: using public_addrv v1:172.21.15.45:6790/0 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: starting mon.c rank -1 at public addrs v1:172.21.15.45:6790/0 at bind addrs v1:172.21.15.45:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(???) e0 preinit fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).mds e1 new map 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: e1 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-14T15:06:21.268 INFO:journalctl@ceph.mon.c.smithi045.stdout: legacy client fscid: -1 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout: 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout: No filesystems configured 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-14T15:06:21.269 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1282523747' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.270 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='client.14184 v1:172.21.15.71:0/1242620952' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Saving service mon spec with placement smithi045:[v1:172.21.15.45:6789]=a;smithi045:[v1:172.21.15.45:6790]=c;smithi071:[v1:172.21.15.71:6789]=b;count:3 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Deploying daemon mon.b on smithi071 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:21.271 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.a calling monitor election 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.b calling monitor election 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: monmap epoch 2 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: last_changed 2024-04-14T15:06:12.472036+0000 2024-04-14T15:06:21.272 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: min_mon_release 19 (squid) 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: election_strategy: 1 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: fsmap 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mgrmap e13: y(active, since 38s) 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: overall HEALTH_OK 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: Deploying daemon mon.c on smithi045 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/3392072406' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:21.273 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:21.274 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 ceph-mon[37795]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-04-14T15:06:21.274 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 bash[37768]: 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 2024-04-14T15:06:21.274 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:20 smithi045 systemd[1]: Started Ceph mon.c for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:06:25.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.962+0000 7fadd5ffb700 1 -- 172.21.15.71:0/754766974 <== mon.1 v1:172.21.15.71:6789/0 11 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1124 (unknown 409899127 0 3811569202) 0x7fadcc017680 con 0x7fadd81074a0 2024-04-14T15:06:25.963 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-14T15:06:25.964 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","modified":"2024-04-14T15:06:20.955860Z","created":"2024-04-14T15:04:37.102829Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6790","nonce":0}]},"addr":"172.21.15.45:6790/0","public_addr":"172.21.15.45:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 >> v1:172.21.15.45:6800/2191709230 conn(0x7fadc803cdb0 legacy=0x7fadc803f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 >> v1:172.21.15.71:6789/0 conn(0x7fadd81074a0 legacy=0x7fadd8195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 shutdown_connections 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 >> 172.21.15.71:0/754766974 conn(0x7fadd80fcf00 msgr2=0x7fadd81094d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 shutdown_connections 2024-04-14T15:06:25.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:25.966+0000 7faddef60700 1 -- 172.21.15.71:0/754766974 wait complete. 2024-04-14T15:06:25.967 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 3 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: mon.a calling monitor election 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: mon.b calling monitor election 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/754766974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: mon.c calling monitor election 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: monmap epoch 3 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: last_changed 2024-04-14T15:06:20.955860+0000 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: min_mon_release 19 (squid) 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: election_strategy: 1 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:26.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: 2: v1:172.21.15.45:6790/0 mon.c 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: fsmap 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: mgrmap e13: y(active, since 47s) 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: overall HEALTH_OK 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: mon.a calling monitor election 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: mon.b calling monitor election 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/754766974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: mon.c calling monitor election 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: monmap epoch 3 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: last_changed 2024-04-14T15:06:20.955860+0000 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: min_mon_release 19 (squid) 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: election_strategy: 1 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: 2: v1:172.21.15.45:6790/0 mon.c 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: fsmap 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: mgrmap e13: y(active, since 47s) 2024-04-14T15:06:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: overall HEALTH_OK 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: mon.a calling monitor election 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: mon.b calling monitor election 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/754766974' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-14T15:06:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: mon.c calling monitor election 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: monmap epoch 3 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: last_changed 2024-04-14T15:06:20.955860+0000 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: created 2024-04-14T15:04:37.102829+0000 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: min_mon_release 19 (squid) 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: election_strategy: 1 2024-04-14T15:06:26.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: 0: v1:172.21.15.45:6789/0 mon.a 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: 1: v1:172.21.15.71:6789/0 mon.b 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: 2: v1:172.21.15.45:6790/0 mon.c 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: fsmap 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: osdmap e4: 0 total, 0 up, 0 in 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: mgrmap e13: y(active, since 47s) 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: overall HEALTH_OK 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:26.522 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-14T15:06:26.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph config generate-minimal-conf 2024-04-14T15:06:27.000 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:27.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:27 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:27.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:27 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:27.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:27 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:27.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:27 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:27 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:27 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:06:27.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:06:27 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:06:27.954+0000 7f4f17a3a700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.488 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.489 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:28.490 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:28.491 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:28 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:28.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:28 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.826+0000 7f2fc040b700 1 -- 172.21.15.45:0/3914214324 >> v1:172.21.15.45:6789/0 conn(0x7f2fb8101530 legacy=0x7f2fb8101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 -- 172.21.15.45:0/3914214324 shutdown_connections 2024-04-14T15:06:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 -- 172.21.15.45:0/3914214324 >> 172.21.15.45:0/3914214324 conn(0x7f2fb80fcda0 msgr2=0x7f2fb80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 -- 172.21.15.45:0/3914214324 shutdown_connections 2024-04-14T15:06:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 -- 172.21.15.45:0/3914214324 wait complete. 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 Processor -- start 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.827+0000 7f2fc040b700 1 -- start start 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2fc040b700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2fb81a94b0 con 0x7f2fb8101530 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2fc040b700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2fb81aa6d0 con 0x7f2fb8199c60 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2fc040b700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f2fb81ab910 con 0x7f2fb81a57e0 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2fbe1a7700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2fb8101530 0x7f2fb8199550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51668/0 (socket says 172.21.15.45:51668) 2024-04-14T15:06:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2fbe1a7700 1 -- 172.21.15.45:0/2918796028 learned_addr learned my addr 172.21.15.45:0/2918796028 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4171440393 0 0) 0x7f2fb81a94b0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa4003bf0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874877782 0 0) 0x7f2fb81aa6d0 con 0x7f2fb8199c60 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb81a94b0 con 0x7f2fb8199c60 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1874592269 0 0) 0x7f2fb81a94b0 con 0x7f2fb8199c60 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2fb81aa6d0 con 0x7f2fb8199c60 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f2fa8001830 con 0x7f2fb8199c60 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.828+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 388789385 0 0) 0x7f2fa4003bf0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2fb81a94b0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f2fb4001ae0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3631429270 0 0) 0x7f2fb81a94b0 con 0x7f2fb8101530 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 >> v1:172.21.15.45:6790/0 conn(0x7f2fb81a57e0 legacy=0x7f2fb81a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 >> v1:172.21.15.71:6789/0 conn(0x7f2fb8199c60 legacy=0x7f2fb81a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb81acb50 con 0x7f2fb8101530 2024-04-14T15:06:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81abb40 con 0x7f2fb8101530 2024-04-14T15:06:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.829+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2fb81ac000 con 0x7f2fb8101530 2024-04-14T15:06:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.830+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb8064110 con 0x7f2fb8101530 2024-04-14T15:06:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.831+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f2fb40011c0 con 0x7f2fb8101530 2024-04-14T15:06:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.831+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f2fb4004c70 con 0x7f2fb8101530 2024-04-14T15:06:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.832+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f2fb4004eb0 con 0x7f2fb8101530 2024-04-14T15:06:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.832+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7f2fb4047ef0 con 0x7f2fb8101530 2024-04-14T15:06:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:28.836+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f2fb4017610 con 0x7f2fb8101530 2024-04-14T15:06:29.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.046+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2fb810bf50 con 0x7f2fb8101530 2024-04-14T15:06:29.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.046+0000 7f2faf7fe700 1 -- 172.21.15.45:0/2918796028 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+190 (unknown 2189217731 0 3600153165) 0x7f2fb4016f10 con 0x7f2fb8101530 2024-04-14T15:06:29.047 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:29.047 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-14T15:06:29.047 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:06:29.048 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = 172.21.15.45:6789/0 172.21.15.71:6789/0 v1:172.21.15.45:6790/0 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.048+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 >> v1:172.21.15.45:6800/2191709230 conn(0x7f2fa403a770 legacy=0x7f2fa403cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.048+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 >> v1:172.21.15.45:6789/0 conn(0x7f2fb8101530 legacy=0x7f2fb8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.048+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 shutdown_connections 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.048+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 >> 172.21.15.45:0/2918796028 conn(0x7f2fb80fcda0 msgr2=0x7f2fb8108d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.048+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 shutdown_connections 2024-04-14T15:06:29.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:29.049+0000 7f2fc040b700 1 -- 172.21.15.45:0/2918796028 wait complete. 2024-04-14T15:06:29.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Reconfiguring mon.a (unknown last config time)... 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: Reconfiguring daemon mon.a on smithi045 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2918796028' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Reconfiguring mon.a (unknown last config time)... 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: Reconfiguring daemon mon.a on smithi045 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2918796028' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Reconfiguring mon.a (unknown last config time)... 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: Reconfiguring daemon mon.a on smithi045 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2918796028' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:29.622 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-14T15:06:29.623 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:06:29.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-04-14T15:06:29.653 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:06:29.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:06:29.720 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:06:29.720 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-04-14T15:06:29.751 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:06:29.751 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:06:29.819 INFO:tasks.cephadm:Adding mgr.y on smithi045 2024-04-14T15:06:29.819 INFO:tasks.cephadm:Adding mgr.x on smithi071 2024-04-14T15:06:29.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply mgr '2;smithi045=y;smithi071=x' 2024-04-14T15:06:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:06:30.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:30 smithi071 ceph-mon[33150]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:30.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:30 smithi045 ceph-mon[29624]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:30.467 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:30 smithi045 ceph-mon[37795]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.560+0000 7f3c5c63a700 1 -- 172.21.15.71:0/3590677975 >> v1:172.21.15.45:6789/0 conn(0x7f3c541025b0 legacy=0x7f3c541029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.560+0000 7f3c5c63a700 1 -- 172.21.15.71:0/3590677975 shutdown_connections 2024-04-14T15:06:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.560+0000 7f3c5c63a700 1 -- 172.21.15.71:0/3590677975 >> 172.21.15.71:0/3590677975 conn(0x7f3c540fdb30 msgr2=0x7f3c540fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.560+0000 7f3c5c63a700 1 -- 172.21.15.71:0/3590677975 shutdown_connections 2024-04-14T15:06:31.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.560+0000 7f3c5c63a700 1 -- 172.21.15.71:0/3590677975 wait complete. 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5c63a700 1 Processor -- start 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5c63a700 1 -- start start 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5c63a700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3c541a2880 con 0x7f3c54197720 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5c63a700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3c541a3aa0 con 0x7f3c541025b0 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5c63a700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f3c541a4ce0 con 0x7f3c5419ec10 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5a3d6700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f3c541025b0 0x7f3c54197010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:45754/0 (socket says 172.21.15.71:45754) 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c5a3d6700 1 -- 172.21.15.71:0/2000742876 learned_addr learned my addr 172.21.15.71:0/2000742876 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46310390 0 0) 0x7f3c541a3aa0 con 0x7f3c541025b0 2024-04-14T15:06:31.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c3c003bf0 con 0x7f3c541025b0 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440936332 0 0) 0x7f3c541a2880 con 0x7f3c54197720 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c541a3aa0 con 0x7f3c54197720 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911819213 0 0) 0x7f3c541a4ce0 con 0x7f3c5419ec10 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c541a2880 con 0x7f3c5419ec10 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.561+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1724386386 0 0) 0x7f3c3c003bf0 con 0x7f3c541025b0 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c541a4ce0 con 0x7f3c541025b0 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f3c4c003960 con 0x7f3c541025b0 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3083306884 0 0) 0x7f3c541a2880 con 0x7f3c5419ec10 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c3c003bf0 con 0x7f3c5419ec10 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f3c44001b50 con 0x7f3c5419ec10 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2730765531 0 0) 0x7f3c541a3aa0 con 0x7f3c54197720 2024-04-14T15:06:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3c541a2880 con 0x7f3c54197720 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f3c480017b0 con 0x7f3c54197720 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1927776460 0 0) 0x7f3c541a4ce0 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 >> v1:172.21.15.45:6790/0 conn(0x7f3c5419ec10 legacy=0x7f3c541a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 >> v1:172.21.15.45:6789/0 conn(0x7f3c54197720 legacy=0x7f3c5419d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c541a5f20 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3c541a2ab0 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f3c541a2ff0 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f3c4c0037b0 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.562+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f3c4c004e80 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.563+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7f3c4c004e80 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.563+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40005150 con 0x7f3c541025b0 2024-04-14T15:06:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.563+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7f3c4c047d70 con 0x7f3c541025b0 2024-04-14T15:06:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.568+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f3c4c017560 con 0x7f3c541025b0 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: Reconfiguring mon.c (monmap changed)... 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:31 smithi071 ceph-mon[33150]: Reconfiguring daemon mon.c on smithi045 2024-04-14T15:06:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: Reconfiguring mon.c (monmap changed)... 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[37795]: Reconfiguring daemon mon.c on smithi045 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: Reconfiguring mon.c (monmap changed)... 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:31 smithi045 ceph-mon[29624]: Reconfiguring daemon mon.c on smithi045 2024-04-14T15:06:31.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.777+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi045=y;smithi071=x", "target": ["mon-mgr", ""]}) -- 0x7f3c40000bc0 con 0x7f3c3c03aa90 2024-04-14T15:06:31.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.783+0000 7f3c537fe700 1 -- 172.21.15.71:0/2000742876 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (unknown 0 0 325935098) 0x7f3c40000bc0 con 0x7f3c3c03aa90 2024-04-14T15:06:31.783 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mgr update... 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 >> v1:172.21.15.45:6800/2191709230 conn(0x7f3c3c03aa90 legacy=0x7f3c3c03cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 >> v1:172.21.15.71:6789/0 conn(0x7f3c541025b0 legacy=0x7f3c54197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 shutdown_connections 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 >> 172.21.15.71:0/2000742876 conn(0x7f3c540fdb30 msgr2=0x7f3c540fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 shutdown_connections 2024-04-14T15:06:31.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:06:31.785+0000 7f3c5c63a700 1 -- 172.21.15.71:0/2000742876 wait complete. 2024-04-14T15:06:32.394 DEBUG:teuthology.orchestra.run.smithi071:mgr.x> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.x.service 2024-04-14T15:06:32.396 INFO:tasks.cephadm:Deploying OSDs... 2024-04-14T15:06:32.396 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:06:32.397 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-14T15:06:32.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-14T15:06:32.413 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-14T15:06:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-14T15:06:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:32.468 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-14T15:06:32.468 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:32.468 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:32.469 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-14 15:06:20.413555666 +0000 2024-04-14T15:06:32.469 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-14 14:59:25.023781822 +0000 2024-04-14T15:06:32.469 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-14 14:59:25.023781822 +0000 2024-04-14T15:06:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-14 14:59:25.023781822 +0000 2024-04-14T15:06:32.469 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-14T15:06:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-14T15:06:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-14T15:06:32.538 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000109081 s, 4.7 MB/s 2024-04-14T15:06:32.540 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-14T15:06:32.597 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-14 15:06:20.413555666 +0000 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-14 14:59:25.359776045 +0000 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-14 14:59:25.359776045 +0000 2024-04-14T15:06:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-14 14:59:25.359776045 +0000 2024-04-14T15:06:32.654 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-14T15:06:32.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-14T15:06:32.719 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-14T15:06:32.719 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000107812 s, 4.7 MB/s 2024-04-14T15:06:32.721 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-14T15:06:32.776 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-14 15:06:20.413555666 +0000 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-14 14:59:25.691770337 +0000 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-14 14:59:25.691770337 +0000 2024-04-14T15:06:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-14 14:59:25.691770337 +0000 2024-04-14T15:06:32.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='client.24106 v1:172.21.15.71:0/2000742876' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi045=y;smithi071=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: Saving service mgr spec with placement smithi045=y;smithi071=x;count:2 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: Reconfiguring mon.b (monmap changed)... 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:32 smithi071 ceph-mon[33150]: Reconfiguring daemon mon.b on smithi071 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='client.24106 v1:172.21.15.71:0/2000742876' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi045=y;smithi071=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: Saving service mgr spec with placement smithi045=y;smithi071=x;count:2 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: Reconfiguring mon.b (monmap changed)... 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:32.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[29624]: Reconfiguring daemon mon.b on smithi071 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='client.24106 v1:172.21.15.71:0/2000742876' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi045=y;smithi071=x", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: Saving service mgr spec with placement smithi045=y;smithi071=x;count:2 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: Reconfiguring mon.b (monmap changed)... 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:32.897 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:32 smithi045 ceph-mon[37795]: Reconfiguring daemon mon.b on smithi071 2024-04-14T15:06:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-14T15:06:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-14T15:06:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 8.5283e-05 s, 6.0 MB/s 2024-04-14T15:06:32.899 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-14T15:06:32.955 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-14 15:06:20.414555650 +0000 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-14 14:59:26.019764697 +0000 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-14 14:59:26.019764697 +0000 2024-04-14T15:06:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-14 14:59:26.019764697 +0000 2024-04-14T15:06:33.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-14T15:06:33.076 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-14T15:06:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-14T15:06:33.077 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103311 s, 5.0 MB/s 2024-04-14T15:06:33.078 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-14T15:06:33.133 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:06:33.133 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-14T15:06:33.149 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-14T15:06:33.149 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:33.203 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-14 15:06:11.269876598 +0000 2024-04-14T15:06:33.204 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-14 14:59:22.424873842 +0000 2024-04-14T15:06:33.204 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-14 14:59:22.424873842 +0000 2024-04-14T15:06:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-14 14:59:22.424873842 +0000 2024-04-14T15:06:33.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-14T15:06:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-14T15:06:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-14T15:06:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000280859 s, 1.8 MB/s 2024-04-14T15:06:33.272 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-14T15:06:33.329 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-14 15:06:11.269876598 +0000 2024-04-14T15:06:33.384 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-14 14:59:22.772867068 +0000 2024-04-14T15:06:33.385 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-14 14:59:22.772867068 +0000 2024-04-14T15:06:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-14 14:59:22.772867068 +0000 2024-04-14T15:06:33.385 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-14T15:06:33.449 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-14T15:06:33.449 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-14T15:06:33.449 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000190497 s, 2.7 MB/s 2024-04-14T15:06:33.450 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-14T15:06:33.506 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-14T15:06:33.561 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-14T15:06:33.561 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-14 15:06:11.270876579 +0000 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-14 14:59:23.110860488 +0000 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-14 14:59:23.110860488 +0000 2024-04-14T15:06:33.562 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-14 14:59:23.110860488 +0000 2024-04-14T15:06:33.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-14T15:06:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-14T15:06:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-14T15:06:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000195958 s, 2.6 MB/s 2024-04-14T15:06:33.630 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-14T15:06:33.688 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-14T15:06:33.744 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-14 15:06:11.270876579 +0000 2024-04-14T15:06:33.745 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-14 14:59:23.461853656 +0000 2024-04-14T15:06:33.746 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-14 14:59:23.461853656 +0000 2024-04-14T15:06:33.746 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-14 14:59:23.461853656 +0000 2024-04-14T15:06:33.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-14T15:06:33.810 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-14T15:06:33.810 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-14T15:06:33.810 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000203328 s, 2.5 MB/s 2024-04-14T15:06:33.812 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-14T15:06:33.867 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-04-14T15:06:33.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-14T15:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:34.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:34.225 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:34.258 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:34.258 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:34.259 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:35.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:34 smithi045 ceph-mon[29624]: Deploying daemon mgr.x on smithi071 2024-04-14T15:06:35.222 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:34 smithi045 ceph-mon[37795]: Deploying daemon mgr.x on smithi071 2024-04-14T15:06:35.239 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:34 smithi071 ceph-mon[33150]: Deploying daemon mgr.x on smithi071 2024-04-14T15:06:35.607 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 systemd[1]: Starting Ceph mgr.x for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:06:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-14T15:06:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-14T15:06:35.852 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-14T15:06:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-14T15:06:35.852 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852788 s, 1.2 GB/s 2024-04-14T15:06:35.853 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-14T15:06:35.903 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 podman[34985]: 2024-04-14 15:06:35.621074025 +0000 UTC m=+0.106666733 container create b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403) 2024-04-14T15:06:35.903 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 podman[34985]: 2024-04-14 15:06:35.532125936 +0000 UTC m=+0.017718600 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:06:35.903 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 podman[34985]: 2024-04-14 15:06:35.819789708 +0000 UTC m=+0.305382356 container init b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:06:35.903 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 podman[34985]: 2024-04-14 15:06:35.822817303 +0000 UTC m=+0.308409958 container start b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:06:35.903 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 bash[34985]: b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 2024-04-14T15:06:36.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:35 smithi045 ceph-mon[29624]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:36.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:35 smithi045 ceph-mon[37795]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:36.357 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:35 smithi071 systemd[1]: Started Ceph mgr.x for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:06:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:35 smithi071 ceph-mon[33150]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:36.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-04-14T15:06:36.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:36.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:36.385+0000 7f44f55d5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:06:36.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:36.482+0000 7f44f55d5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:06:37.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.139 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:37.262 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.167+0000 7f44f55d5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.262+0000 7f44f55d5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:37.263 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:37.514 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.407+0000 7f44f55d5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:06:37.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.514+0000 7f44f55d5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:06:37.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.600+0000 7f44f55d5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:06:38.149 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.861+0000 7f44f55d5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:06:38.150 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:37.971+0000 7f44f55d5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:06:38.150 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:38.055+0000 7f44f55d5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:06:38.150 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:38.149+0000 7f44f55d5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:06:38.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.299+0000 7fe192e4d700 1 -- 172.21.15.45:0/2507827952 >> v1:172.21.15.45:6790/0 conn(0x7fe18c10aa60 legacy=0x7fe18c10cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:38.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.299+0000 7fe192e4d700 1 -- 172.21.15.45:0/2507827952 shutdown_connections 2024-04-14T15:06:38.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.299+0000 7fe192e4d700 1 -- 172.21.15.45:0/2507827952 >> 172.21.15.45:0/2507827952 conn(0x7fe18c0fdeb0 msgr2=0x7fe18c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.299+0000 7fe192e4d700 1 -- 172.21.15.45:0/2507827952 shutdown_connections 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.299+0000 7fe192e4d700 1 -- 172.21.15.45:0/2507827952 wait complete. 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe192e4d700 1 Processor -- start 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe192e4d700 1 -- start start 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe192e4d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe18c198a60 con 0x7fe18c106d20 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe192e4d700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe18c1a4250 con 0x7fe18c1028f0 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe192e4d700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe18c1a5430 con 0x7fe18c10aa60 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe1913ea700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fe18c10aa60 0x7fe18c1a1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:40900/0 (socket says 172.21.15.45:40900) 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe1913ea700 1 -- 172.21.15.45:0/1933915445 learned_addr learned my addr 172.21.15.45:0/1933915445 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.300+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244612870 0 0) 0x7fe18c1a5430 con 0x7fe18c10aa60 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe17c0035b0 con 0x7fe18c10aa60 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338774931 0 0) 0x7fe18c1a4250 con 0x7fe18c1028f0 2024-04-14T15:06:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe18c1a5430 con 0x7fe18c1028f0 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3383956290 0 0) 0x7fe17c0035b0 con 0x7fe18c10aa60 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe18c1a4250 con 0x7fe18c10aa60 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832192045 0 0) 0x7fe18c198a60 con 0x7fe18c106d20 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe17c0035b0 con 0x7fe18c106d20 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe1840047b0 con 0x7fe18c10aa60 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2302047167 0 0) 0x7fe18c1a5430 con 0x7fe18c1028f0 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe18c198a60 con 0x7fe18c1028f0 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe180001830 con 0x7fe18c1028f0 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1435824136 0 0) 0x7fe18c1a4250 con 0x7fe18c10aa60 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 >> v1:172.21.15.71:6789/0 conn(0x7fe18c1028f0 legacy=0x7fe18c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 >> v1:172.21.15.45:6789/0 conn(0x7fe18c106d20 legacy=0x7fe18c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1a6650 con 0x7fe18c10aa60 2024-04-14T15:06:38.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c1a4480 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fe184001e30 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe184005480 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.301+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe18c1a49c0 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.302+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c1116f0 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.302+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 13) ==== 46579+0+0 (unknown 3457803259 0 0) 0x7fe184005620 con 0x7fe18c10aa60 2024-04-14T15:06:38.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.303+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(4..4 src has 1..4) ==== 1069+0+0 (unknown 2947844594 0 0) 0x7fe184048350 con 0x7fe18c10aa60 2024-04-14T15:06:38.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.308+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fe184017980 con 0x7fe18c10aa60 2024-04-14T15:06:38.475 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:38.327+0000 7f44f55d5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:06:38.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:38.533+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe18c1a4d00 con 0x7fe17c03a830 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:38.706 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:38.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-14T15:06:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: Reconfiguring mgr.y (unknown last config time)... 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: Reconfiguring daemon mgr.y on smithi045 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:39.299+0000 7f44f55d5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:06:39.608 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:39.428+0000 7f44f55d5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:06:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: Reconfiguring mgr.y (unknown last config time)... 2024-04-14T15:06:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: Reconfiguring daemon mgr.y on smithi045 2024-04-14T15:06:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:06:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: Reconfiguring mgr.y (unknown last config time)... 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: Reconfiguring daemon mgr.y on smithi045 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.048 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:39.765+0000 7f44f55d5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:06:40.048 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:39.853+0000 7f44f55d5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:06:40.048 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:40.047+0000 7f44f55d5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:06:40.357 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:40.235+0000 7f44f55d5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:06:40.682 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:40.682 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='client.34112 v1:172.21.15.45:0/1933915445' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:40.682 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='client.34112 v1:172.21.15.45:0/1933915445' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:40.683 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:40.684 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:40 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='client.34112 v1:172.21.15.45:0/1933915445' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:06:40.777 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:40.778 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:40.519+0000 7f44f55d5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:06:40.778 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:40.777+0000 7f44f55d5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:06:41.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:41.012+0000 7f44f55d5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:06:42.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:41.808+0000 7f44f55d5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:06:42.691 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]: dispatch 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]': finished 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: osdmap e5: 1 total, 0 up, 1 in 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1843368098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:42.377+0000 7f44f55d5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:42.474+0000 7f44f55d5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:06:42.692 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:06:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:06:42.691+0000 7f44f55d5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]: dispatch 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]': finished 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: osdmap e5: 1 total, 0 up, 1 in 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1843368098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]: dispatch 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2259078423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceb62cce-7685-4b96-930f-bba7be80eabd"}]': finished 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: osdmap e5: 1 total, 0 up, 1 in 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1843368098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:06:43.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:43.488+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mgrmap(e 14) ==== 92567+0+0 (unknown 4202890743 0 0) 0x7fe184046bf0 con 0x7fe18c10aa60 2024-04-14T15:06:43.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[29624]: Standby manager daemon x started 2024-04-14T15:06:43.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:06:43.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:06:43.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[37795]: Standby manager daemon x started 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:06:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:43 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:06:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:43 smithi071 ceph-mon[33150]: Standby manager daemon x started 2024-04-14T15:06:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:43 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:06:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:43 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:06:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:43 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:06:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:43 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3624058075' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:06:44.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[29624]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:44.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[29624]: mgrmap e14: y(active, since 64s), standbys: x 2024-04-14T15:06:44.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:06:44.701 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[37795]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:44.701 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[37795]: mgrmap e14: y(active, since 64s), standbys: x 2024-04-14T15:06:44.701 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:44 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:06:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:44 smithi071 ceph-mon[33150]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:44 smithi071 ceph-mon[33150]: mgrmap e14: y(active, since 64s), standbys: x 2024-04-14T15:06:44.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:44 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:06:46.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:46 smithi045 ceph-mon[29624]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:46.596 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:46 smithi045 ceph-mon[37795]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:46 smithi071 ceph-mon[33150]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[29624]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-14T15:06:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[37795]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-14T15:06:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:48 smithi071 ceph-mon[33150]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:48.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-14T15:06:48.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:06:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:49 smithi045 ceph-mon[29624]: Deploying daemon osd.0 on smithi045 2024-04-14T15:06:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:49 smithi045 ceph-mon[37795]: Deploying daemon osd.0 on smithi045 2024-04-14T15:06:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:49 smithi071 ceph-mon[33150]: Deploying daemon osd.0 on smithi045 2024-04-14T15:06:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:50 smithi045 ceph-mon[29624]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:50 smithi045 ceph-mon[37795]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:50 smithi071 ceph-mon[33150]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:51.768 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:51 smithi045 ceph-mon[29624]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:51.768 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:51 smithi045 ceph-mon[37795]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:51 smithi071 ceph-mon[33150]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:54.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[29624]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[37795]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.166 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:53 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:53 smithi071 ceph-mon[33150]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:53 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:53 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:53 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:06:55.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.982+0000 7fe189ffb700 1 -- 172.21.15.45:0/1933915445 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 1546445030) 0x7fe18c1a4d00 con 0x7fe17c03a830 2024-04-14T15:06:55.983 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-04-14T15:06:55.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 >> v1:172.21.15.45:6800/2191709230 conn(0x7fe17c03a830 legacy=0x7fe17c03ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:55.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 >> v1:172.21.15.45:6790/0 conn(0x7fe18c10aa60 legacy=0x7fe18c1a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:06:55.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 shutdown_connections 2024-04-14T15:06:55.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 >> 172.21.15.45:0/1933915445 conn(0x7fe18c0fdeb0 msgr2=0x7fe18c109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:06:55.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 shutdown_connections 2024-04-14T15:06:55.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:06:55.985+0000 7fe192e4d700 1 -- 172.21.15.45:0/1933915445 wait complete. 2024-04-14T15:06:56.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:55 smithi045 ceph-mon[29624]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:55 smithi045 ceph-mon[37795]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:55 smithi071 ceph-mon[33150]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:56.553 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.0.service 2024-04-14T15:06:56.556 INFO:tasks.cephadm:Deploying osd.1 on smithi045 with /dev/vg_nvme/lv_3... 2024-04-14T15:06:56.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-14T15:06:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:06:57.240 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.241 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.242 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:06:58.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:58.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: osdmap e6: 1 total, 0 up, 1 in 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[29624]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: osdmap e6: 1 total, 0 up, 1 in 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:58.227 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-mon[37795]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:58.228 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:06:57.964+0000 7f4e86cdb700 -1 osd.0 0 waiting for initial osdmap 2024-04-14T15:06:58.228 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:06:57 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:06:57.967+0000 7f4e82ace700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: osdmap e6: 1 total, 0 up, 1 in 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: from='osd.0 v1:172.21.15.45:6801/609845118' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:57 smithi071 ceph-mon[33150]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:06:59.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[29624]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:06:59.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[29624]: osdmap e7: 1 total, 0 up, 1 in 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[37795]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[37795]: osdmap e7: 1 total, 0 up, 1 in 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:58 smithi071 ceph-mon[33150]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:06:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:58 smithi071 ceph-mon[33150]: osdmap e7: 1 total, 0 up, 1 in 2024-04-14T15:06:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127525 s, 822 MB/s 2024-04-14T15:06:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-14T15:07:00.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: osd.0 v1:172.21.15.45:6801/609845118 boot 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: osdmap e8: 1 total, 1 up, 1 in 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: osd.0 v1:172.21.15.45:6801/609845118 boot 2024-04-14T15:07:00.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: osdmap e8: 1 total, 1 up, 1 in 2024-04-14T15:07:00.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:06:59 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: osd.0 v1:172.21.15.45:6801/609845118 boot 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: osdmap e8: 1 total, 1 up, 1 in 2024-04-14T15:07:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:06:59 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:07:00.989 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:07:01.245 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[37795]: osdmap e9: 1 total, 1 up, 1 in 2024-04-14T15:07:01.245 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.245 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.245 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:01.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[29624]: osdmap e9: 1 total, 1 up, 1 in 2024-04-14T15:07:01.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:00 smithi071 ceph-mon[33150]: osdmap e9: 1 total, 1 up, 1 in 2024-04-14T15:07:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:02.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:02.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi045 2024-04-14T15:07:02.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 16069M 2024-04-14T15:07:02.162 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi045 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 16069M 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.163 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.164 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi045 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 16069M 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:02.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:02.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:02.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.445+0000 7fe5765cd700 1 -- 172.21.15.45:0/3046464819 >> v1:172.21.15.45:6789/0 conn(0x7fe570109950 legacy=0x7fe57010be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.445+0000 7fe5765cd700 1 -- 172.21.15.45:0/3046464819 shutdown_connections 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.445+0000 7fe5765cd700 1 -- 172.21.15.45:0/3046464819 >> 172.21.15.45:0/3046464819 conn(0x7fe5700fcd80 msgr2=0x7fe5700ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.445+0000 7fe5765cd700 1 -- 172.21.15.45:0/3046464819 shutdown_connections 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.445+0000 7fe5765cd700 1 -- 172.21.15.45:0/3046464819 wait complete. 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe5765cd700 1 Processor -- start 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe5765cd700 1 -- start start 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe5765cd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe57006b120 con 0x7fe5701017e0 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe5765cd700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe57006b350 con 0x7fe570109950 2024-04-14T15:07:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe5765cd700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe5701a7490 con 0x7fe570105c10 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe56ffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5701017e0 0x7fe57019ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43856/0 (socket says 172.21.15.45:43856) 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe56ffff700 1 -- 172.21.15.45:0/201664381 learned_addr learned my addr 172.21.15.45:0/201664381 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 591401116 0 0) 0x7fe5701a7490 con 0x7fe570105c10 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5580035b0 con 0x7fe570105c10 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.446+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1828125755 0 0) 0x7fe57006b120 con 0x7fe5701017e0 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5701a7490 con 0x7fe5701017e0 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3684907203 0 0) 0x7fe57006b350 con 0x7fe570109950 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe57006b120 con 0x7fe570109950 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1390800507 0 0) 0x7fe5701a7490 con 0x7fe5701017e0 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57006b350 con 0x7fe5701017e0 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe5600018b0 con 0x7fe5701017e0 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1095685653 0 0) 0x7fe5580035b0 con 0x7fe570105c10 2024-04-14T15:07:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5701a7490 con 0x7fe570105c10 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe564001890 con 0x7fe570105c10 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1344483174 0 0) 0x7fe57006b120 con 0x7fe570109950 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5580035b0 con 0x7fe570109950 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe568001b80 con 0x7fe570109950 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4217598783 0 0) 0x7fe57006b350 con 0x7fe5701017e0 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 >> v1:172.21.15.45:6790/0 conn(0x7fe570105c10 legacy=0x7fe5701a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 >> v1:172.21.15.71:6789/0 conn(0x7fe570109950 legacy=0x7fe5701a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5701a86b0 con 0x7fe5701017e0 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe5701a76c0 con 0x7fe5701017e0 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe5701a7c00 con 0x7fe5701017e0 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fe560001cc0 con 0x7fe5701017e0 2024-04-14T15:07:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.447+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe560005d30 con 0x7fe5701017e0 2024-04-14T15:07:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.448+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c005150 con 0x7fe5701017e0 2024-04-14T15:07:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.449+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 14) ==== 92567+0+0 (unknown 4202890743 0 0) 0x7fe560004070 con 0x7fe5701017e0 2024-04-14T15:07:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.450+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(9..9 src has 1..9) ==== 1541+0+0 (unknown 66663150 0 0) 0x7fe560088d10 con 0x7fe5701017e0 2024-04-14T15:07:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.454+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fe5600b9fa0 con 0x7fe5701017e0 2024-04-14T15:07:02.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:02.672+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe55c000bc0 con 0x7fe55806f830 2024-04-14T15:07:03.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:03.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:03.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:03.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:03.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:03.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:04.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:03 smithi045 ceph-mon[29624]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:04.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:03 smithi045 ceph-mon[29624]: from='client.14244 v1:172.21.15.45:0/201664381' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:04.212 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:03 smithi045 ceph-mon[37795]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:04.212 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:03 smithi045 ceph-mon[37795]: from='client.14244 v1:172.21.15.45:0/201664381' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:03 smithi071 ceph-mon[33150]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:03 smithi071 ceph-mon[33150]: from='client.14244 v1:172.21.15.45:0/201664381' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4014532584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]': finished 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: osdmap e10: 2 total, 1 up, 2 in 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/978585841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:06.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4014532584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]': finished 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: osdmap e10: 2 total, 1 up, 2 in 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:06.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/978585841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4014532584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: from='client.34145 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]: dispatch 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: from='client.34145 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39c5f730-a033-4475-9c37-37fe6929650d"}]': finished 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: osdmap e10: 2 total, 1 up, 2 in 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/978585841' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:08.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:07 smithi045 ceph-mon[29624]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:08.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:07 smithi045 ceph-mon[37795]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:07 smithi071 ceph-mon[33150]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:10.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:09 smithi045 ceph-mon[29624]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:10.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:09 smithi045 ceph-mon[37795]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:09 smithi071 ceph-mon[33150]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:12.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[29624]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[29624]: Deploying daemon osd.1 on smithi045 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[37795]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:12.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:11 smithi045 ceph-mon[37795]: Deploying daemon osd.1 on smithi045 2024-04-14T15:07:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:11 smithi071 ceph-mon[33150]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:11 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-14T15:07:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:11 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:11 smithi071 ceph-mon[33150]: Deploying daemon osd.1 on smithi045 2024-04-14T15:07:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:13 smithi045 ceph-mon[29624]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:13 smithi045 ceph-mon[37795]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:13 smithi071 ceph-mon[33150]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:15 smithi045 ceph-mon[29624]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:16.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:15 smithi045 ceph-mon[37795]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:15 smithi071 ceph-mon[33150]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[37795]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[29624]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:17 smithi071 ceph-mon[33150]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:20.152 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 1 on host 'smithi045' 2024-04-14T15:07:20.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.151+0000 7fe56d7fa700 1 -- 172.21.15.45:0/201664381 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3739218) 0x7fe55c000bc0 con 0x7fe55806f830 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.153+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 >> v1:172.21.15.45:6800/2191709230 conn(0x7fe55806f830 legacy=0x7fe558071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.153+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 >> v1:172.21.15.45:6789/0 conn(0x7fe5701017e0 legacy=0x7fe57019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.154+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 shutdown_connections 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.154+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 >> 172.21.15.45:0/201664381 conn(0x7fe5700fcd80 msgr2=0x7fe5701087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.154+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 shutdown_connections 2024-04-14T15:07:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:20.154+0000 7fe5765cd700 1 -- 172.21.15.45:0/201664381 wait complete. 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.735 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:20 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:20.760 DEBUG:teuthology.orchestra.run.smithi045:osd.1> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.1.service 2024-04-14T15:07:20.762 INFO:tasks.cephadm:Deploying osd.2 on smithi045 with /dev/vg_nvme/lv_2... 2024-04-14T15:07:20.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:20.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:20 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:21.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[29624]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[29624]: osdmap e11: 2 total, 1 up, 2 in 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[29624]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[37795]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[37795]: osdmap e11: 2 total, 1 up, 2 in 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[37795]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:21.690 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:21.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:21 smithi071 ceph-mon[33150]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-14T15:07:21.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:21 smithi071 ceph-mon[33150]: osdmap e11: 2 total, 1 up, 2 in 2024-04-14T15:07:21.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:21 smithi071 ceph-mon[33150]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:21.741 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:21 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:21.986 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:07:21.688+0000 7ff0c3f1c700 -1 osd.1 0 waiting for initial osdmap 2024-04-14T15:07:21.986 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:21 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:07:21.691+0000 7ff0bbd07700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:07:22.274 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:07:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: osdmap e12: 2 total, 1 up, 2 in 2024-04-14T15:07:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[29624]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: osdmap e12: 2 total, 1 up, 2 in 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.642 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:22 smithi045 ceph-mon[37795]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: osdmap e12: 2 total, 1 up, 2 in 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:22.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:22 smithi071 ceph-mon[33150]: from='osd.1 v1:172.21.15.45:6805/1698650141' entity='osd.1' 2024-04-14T15:07:23.739 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: osd.1 v1:172.21.15.45:6805/1698650141 boot 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: osdmap e13: 2 total, 2 up, 2 in 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: osd.1 v1:172.21.15.45:6805/1698650141 boot 2024-04-14T15:07:23.740 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: osdmap e13: 2 total, 2 up, 2 in 2024-04-14T15:07:23.741 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:23 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: osd.1 v1:172.21.15.45:6805/1698650141 boot 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: osdmap e13: 2 total, 2 up, 2 in 2024-04-14T15:07:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:23 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:07:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-14T15:07:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-14T15:07:24.667 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-14T15:07:24.667 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-14T15:07:24.667 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00942334 s, 1.1 GB/s 2024-04-14T15:07:24.668 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:24.668 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:24.669 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:25.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-04-14T15:07:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi045 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 8034M 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[37795]: osdmap e14: 2 total, 2 up, 2 in 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi045 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 8034M 2024-04-14T15:07:25.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:25 smithi045 ceph-mon[29624]: osdmap e14: 2 total, 2 up, 2 in 2024-04-14T15:07:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:25 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi045 2024-04-14T15:07:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:25 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 8034M 2024-04-14T15:07:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:25 smithi071 ceph-mon[33150]: osdmap e14: 2 total, 2 up, 2 in 2024-04-14T15:07:26.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:26 smithi045 ceph-mon[29624]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:26.541 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:26 smithi045 ceph-mon[37795]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:26.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:26 smithi071 ceph-mon[33150]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.063+0000 7f6be879e700 1 -- 172.21.15.45:0/3428476180 >> v1:172.21.15.45:6790/0 conn(0x7f6be01017d0 legacy=0x7f6be0101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.063+0000 7f6be879e700 1 -- 172.21.15.45:0/3428476180 shutdown_connections 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.063+0000 7f6be879e700 1 -- 172.21.15.45:0/3428476180 >> 172.21.15.45:0/3428476180 conn(0x7f6be00fcd90 msgr2=0x7f6be00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.063+0000 7f6be879e700 1 -- 172.21.15.45:0/3428476180 shutdown_connections 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.063+0000 7f6be879e700 1 -- 172.21.15.45:0/3428476180 wait complete. 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be879e700 1 Processor -- start 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be879e700 1 -- start start 2024-04-14T15:07:27.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be879e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6be006bb10 con 0x7f6be0105c00 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be879e700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6be006bd40 con 0x7f6be0109940 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be879e700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6be01a7500 con 0x7f6be01017d0 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be5d39700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6be0105c00 0x7f6be01a0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50620/0 (socket says 172.21.15.45:50620) 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be653a700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6be01017d0 0x7f6be019cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:59136/0 (socket says 172.21.15.45:59136) 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6be5d39700 1 -- 172.21.15.45:0/628819508 learned_addr learned my addr 172.21.15.45:0/628819508 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545343039 0 0) 0x7f6be01a7500 con 0x7f6be01017d0 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc80035b0 con 0x7f6be01017d0 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859939646 0 0) 0x7f6be006bb10 con 0x7f6be0105c00 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6be01a7500 con 0x7f6be0105c00 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169590613 0 0) 0x7f6be006bd40 con 0x7f6be0109940 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.064+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6be006bb10 con 0x7f6be0109940 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2844220088 0 0) 0x7f6bc80035b0 con 0x7f6be01017d0 2024-04-14T15:07:27.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6be006bd40 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1332030899 0 0) 0x7f6be01a7500 con 0x7f6be0105c00 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6bc80035b0 con 0x7f6be0105c00 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6bd00046f0 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6bd4001940 con 0x7f6be0105c00 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 26854574 0 0) 0x7f6be006bb10 con 0x7f6be0109940 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6be01a7500 con 0x7f6be0109940 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6bd8001130 con 0x7f6be0109940 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 725981353 0 0) 0x7f6be006bd40 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 >> v1:172.21.15.71:6789/0 conn(0x7f6be0109940 legacy=0x7f6be01a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 >> v1:172.21.15.45:6789/0 conn(0x7f6be0105c00 legacy=0x7f6be01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be01a8720 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6be01a7730 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6be01a7c70 con 0x7f6be01017d0 2024-04-14T15:07:27.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6bd0001e60 con 0x7f6be01017d0 2024-04-14T15:07:27.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.065+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6bd00050a0 con 0x7f6be01017d0 2024-04-14T15:07:27.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.066+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc005150 con 0x7f6be01017d0 2024-04-14T15:07:27.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.066+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 14) ==== 92567+0+0 (unknown 4202890743 0 0) 0x7f6bd0003a90 con 0x7f6be01017d0 2024-04-14T15:07:27.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.067+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(14..14 src has 1..14) ==== 1833+0+0 (unknown 2223371577 0 0) 0x7f6bd0089230 con 0x7f6be01017d0 2024-04-14T15:07:27.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.072+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f6bd0057720 con 0x7f6be01017d0 2024-04-14T15:07:27.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:27.287+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6bcc000bc0 con 0x7f6bc806f910 2024-04-14T15:07:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:27 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:27 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:27 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:27 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:28.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:28 smithi045 ceph-mon[29624]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:28.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:28 smithi045 ceph-mon[29624]: from='client.34166 v1:172.21.15.45:0/628819508' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:28.726 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:28 smithi045 ceph-mon[37795]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:28.726 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:28 smithi045 ceph-mon[37795]: from='client.34166 v1:172.21.15.45:0/628819508' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:28 smithi071 ceph-mon[33150]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:28 smithi071 ceph-mon[33150]: from='client.34166 v1:172.21.15.45:0/628819508' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:30.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/171709508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]': finished 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: osdmap e15: 3 total, 2 up, 3 in 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/171709508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]': finished 2024-04-14T15:07:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: osdmap e15: 3 total, 2 up, 3 in 2024-04-14T15:07:30.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:30 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/171709508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]: dispatch 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11bba3d9-26dc-4989-ae9d-555847390c3c"}]': finished 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: osdmap e15: 3 total, 2 up, 3 in 2024-04-14T15:07:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:30 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1935220951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1935220951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1935220951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:32.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:32 smithi045 ceph-mon[29624]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:32.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:32 smithi045 ceph-mon[37795]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:32 smithi071 ceph-mon[33150]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:34 smithi045 ceph-mon[29624]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:34 smithi045 ceph-mon[37795]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:34.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:34 smithi071 ceph-mon[33150]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:36.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[29624]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-14T15:07:36.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:36.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[37795]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:36.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-14T15:07:36.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:36 smithi071 ceph-mon[33150]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-14T15:07:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:37.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:37 smithi045 ceph-mon[29624]: Deploying daemon osd.2 on smithi045 2024-04-14T15:07:37.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:37 smithi045 ceph-mon[37795]: Deploying daemon osd.2 on smithi045 2024-04-14T15:07:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:37 smithi071 ceph-mon[33150]: Deploying daemon osd.2 on smithi045 2024-04-14T15:07:38.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:38 smithi045 ceph-mon[29624]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:38.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:38 smithi045 ceph-mon[37795]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:38 smithi071 ceph-mon[33150]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:40.691 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:40 smithi045 ceph-mon[29624]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:40.691 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:40 smithi045 ceph-mon[37795]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:40 smithi071 ceph-mon[33150]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:42.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[29624]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[37795]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:42 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:42 smithi071 ceph-mon[33150]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:42 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:42 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:42 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:44 smithi045 ceph-mon[29624]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:44 smithi045 ceph-mon[37795]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:44.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.839+0000 7f6bdf7fe700 1 -- 172.21.15.45:0/628819508 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3825679758) 0x7f6bcc000bc0 con 0x7f6bc806f910 2024-04-14T15:07:44.840 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 2 on host 'smithi045' 2024-04-14T15:07:44.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 >> v1:172.21.15.45:6800/2191709230 conn(0x7f6bc806f910 legacy=0x7f6bc8071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:44.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 >> v1:172.21.15.45:6790/0 conn(0x7f6be01017d0 legacy=0x7f6be019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:44.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 shutdown_connections 2024-04-14T15:07:44.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 >> 172.21.15.45:0/628819508 conn(0x7f6be00fcd90 msgr2=0x7f6be010c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:07:44.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 shutdown_connections 2024-04-14T15:07:44.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:44.842+0000 7f6be879e700 1 -- 172.21.15.45:0/628819508 wait complete. 2024-04-14T15:07:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:44 smithi071 ceph-mon[33150]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:45.513 DEBUG:teuthology.orchestra.run.smithi045:osd.2> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.2.service 2024-04-14T15:07:45.516 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_1... 2024-04-14T15:07:45.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-14T15:07:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[29624]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[29624]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[37795]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[37795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:45 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:45.805 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:07:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:45 smithi071 ceph-mon[33150]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:45 smithi071 ceph-mon[33150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-14T15:07:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:45 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:45 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: osdmap e16: 3 total, 2 up, 3 in 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: osdmap e16: 3 total, 2 up, 3 in 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.748 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2[53733]: 2024-04-14T15:07:46.490+0000 7fe4377ba700 -1 osd.2 0 waiting for initial osdmap 2024-04-14T15:07:46.748 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2[53733]: 2024-04-14T15:07:46.493+0000 7fe4335ad700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:07:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-14T15:07:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: osdmap e16: 3 total, 2 up, 3 in 2024-04-14T15:07:46.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='osd.2 v1:172.21.15.45:6809/1422326616' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:46.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:07:46.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:46.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:46 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:47.714 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[37795]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:47.714 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[37795]: osdmap e17: 3 total, 2 up, 3 in 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[29624]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[29624]: osdmap e17: 3 total, 2 up, 3 in 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:47 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:47 smithi071 ceph-mon[33150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:07:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:47 smithi071 ceph-mon[33150]: osdmap e17: 3 total, 2 up, 3 in 2024-04-14T15:07:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:47 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:47 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:47 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00942039 s, 1.1 GB/s 2024-04-14T15:07:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-14T15:07:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: osd.2 v1:172.21.15.45:6809/1422326616 boot 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: osdmap e18: 3 total, 3 up, 3 in 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: osd.2 v1:172.21.15.45:6809/1422326616 boot 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: osdmap e18: 3 total, 3 up, 3 in 2024-04-14T15:07:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:48.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-04-14T15:07:48.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:07:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:07:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-14T15:07:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: osd.2 v1:172.21.15.45:6809/1422326616 boot 2024-04-14T15:07:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: osdmap e18: 3 total, 3 up, 3 in 2024-04-14T15:07:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:07:49.206 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:07:49.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:49 smithi045 ceph-mon[37795]: osdmap e19: 3 total, 3 up, 3 in 2024-04-14T15:07:49.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:49 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:49 smithi045 ceph-mon[29624]: osdmap e19: 3 total, 3 up, 3 in 2024-04-14T15:07:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:49 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:49 smithi071 ceph-mon[33150]: osdmap e19: 3 total, 3 up, 3 in 2024-04-14T15:07:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:49 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:50.670 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57464]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57464]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57426]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57426]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57426]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57426]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57429]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57429]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57429]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.671 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57429]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: osdmap e20: 3 total, 3 up, 3 in 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi045 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.672 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 5356M 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: osdmap e20: 3 total, 3 up, 3 in 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi045 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 5356M 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:50.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: pgmap v74: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: osdmap e20: 3 total, 3 up, 3 in 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi045 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 5356M 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:50.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:50.926 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57467]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:50.926 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57467]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.926 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57467]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.926 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57467]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57482]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.927 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57485]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.927 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57470]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57470]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57470]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57470]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57473]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57473]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57473]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.928 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57473]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57476]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57476]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57476]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57476]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:50.928 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:50.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:50.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:50 smithi045 sudo[57479]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35719]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35719]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35719]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35722]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35722]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35722]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-14T15:07:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:50 smithi071 sudo[35722]: pam_unix(sudo:session): session closed for user root 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.388+0000 7f07e64cc700 1 -- 172.21.15.45:0/1859214901 >> v1:172.21.15.45:6790/0 conn(0x7f07e0105c10 legacy=0x7f07e0108060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.388+0000 7f07e64cc700 1 -- 172.21.15.45:0/1859214901 shutdown_connections 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.388+0000 7f07e64cc700 1 -- 172.21.15.45:0/1859214901 >> 172.21.15.45:0/1859214901 conn(0x7f07e00fcd80 msgr2=0x7f07e00ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.388+0000 7f07e64cc700 1 -- 172.21.15.45:0/1859214901 shutdown_connections 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.388+0000 7f07e64cc700 1 -- 172.21.15.45:0/1859214901 wait complete. 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07e64cc700 1 Processor -- start 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07e64cc700 1 -- start start 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07e64cc700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f07e01a4ec0 con 0x7f07e01017e0 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07e64cc700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f07e01a6100 con 0x7f07e0109950 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07e64cc700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f07e01a7340 con 0x7f07e0105c10 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07dffff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f07e01017e0 0x7f07e0107ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47648/0 (socket says 172.21.15.45:47648) 2024-04-14T15:07:51.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07dffff700 1 -- 172.21.15.45:0/4215527550 learned_addr learned my addr 172.21.15.45:0/4215527550 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3434135370 0 0) 0x7f07e01a7340 con 0x7f07e0105c10 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07cc0050b0 con 0x7f07e0105c10 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1773566313 0 0) 0x7f07e01a4ec0 con 0x7f07e01017e0 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e01a7340 con 0x7f07e01017e0 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2487218311 0 0) 0x7f07e01a6100 con 0x7f07e0109950 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.389+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e01a4ec0 con 0x7f07e0109950 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1338180597 0 0) 0x7f07cc0050b0 con 0x7f07e0105c10 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f07e01a6100 con 0x7f07e0105c10 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f07c8004500 con 0x7f07e0105c10 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1913200928 0 0) 0x7f07e01a7340 con 0x7f07e01017e0 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f07cc0050b0 con 0x7f07e01017e0 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f07d0001890 con 0x7f07e01017e0 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4118114265 0 0) 0x7f07e01a4ec0 con 0x7f07e0109950 2024-04-14T15:07:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f07e01a7340 con 0x7f07e0109950 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f07d8001af0 con 0x7f07e0109950 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 685498271 0 0) 0x7f07e01a6100 con 0x7f07e0105c10 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 >> v1:172.21.15.71:6789/0 conn(0x7f07e0109950 legacy=0x7f07e01a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 >> v1:172.21.15.45:6789/0 conn(0x7f07e01017e0 legacy=0x7f07e0107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01a8580 con 0x7f07e0105c10 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f07e01a50f0 con 0x7f07e0105c10 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f07e01a5630 con 0x7f07e0105c10 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f07c8001b10 con 0x7f07e0105c10 2024-04-14T15:07:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.390+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f07c8004de0 con 0x7f07e0105c10 2024-04-14T15:07:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.391+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c0005150 con 0x7f07e0105c10 2024-04-14T15:07:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.391+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 14) ==== 92567+0+0 (unknown 4202890743 0 0) 0x7f07c801b780 con 0x7f07e0105c10 2024-04-14T15:07:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.392+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(21..21 src has 1..21) ==== 2511+0+0 (unknown 3742696093 0 0) 0x7f07c8088040 con 0x7f07e0105c10 2024-04-14T15:07:51.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.397+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f07c8059f60 con 0x7f07e0105c10 2024-04-14T15:07:51.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:51.617+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f07c0000bc0 con 0x7f07cc06f800 2024-04-14T15:07:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: osdmap e21: 3 total, 3 up, 3 in 2024-04-14T15:07:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:51 smithi071 ceph-mon[33150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: osdmap e21: 3 total, 3 up, 3 in 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[37795]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: osdmap e21: 3 total, 3 up, 3 in 2024-04-14T15:07:51.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:51.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:07:52.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:07:52.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:07:52.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-14T15:07:52.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:07:52.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:51 smithi045 ceph-mon[29624]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-14T15:07:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:52 smithi071 ceph-mon[33150]: osdmap e22: 3 total, 3 up, 3 in 2024-04-14T15:07:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:52 smithi071 ceph-mon[33150]: from='client.34202 v1:172.21.15.45:0/4215527550' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:52 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:52 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:52 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[37795]: osdmap e22: 3 total, 3 up, 3 in 2024-04-14T15:07:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[37795]: from='client.34202 v1:172.21.15.45:0/4215527550' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[29624]: osdmap e22: 3 total, 3 up, 3 in 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[29624]: from='client.34202 v1:172.21.15.45:0/4215527550' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:07:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:52 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:07:53.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:07:53.521+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f07c8051950 con 0x7f07e0105c10 2024-04-14T15:07:53.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:53 smithi045 ceph-mon[29624]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:53.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:53 smithi045 ceph-mon[37795]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:53 smithi071 ceph-mon[33150]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: mgrmap e15: y(active, since 2m), standbys: x 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2660011856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: from='client.34208 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: from='client.34208 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]': finished 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: osdmap e23: 4 total, 3 up, 4 in 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: mgrmap e15: y(active, since 2m), standbys: x 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2660011856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.770 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: from='client.34208 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.771 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: from='client.34208 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]': finished 2024-04-14T15:07:54.771 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: osdmap e23: 4 total, 3 up, 4 in 2024-04-14T15:07:54.771 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:54 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:07:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: mgrmap e15: y(active, since 2m), standbys: x 2024-04-14T15:07:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2660011856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: from='client.34208 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]: dispatch 2024-04-14T15:07:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: from='client.34208 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ce0a73-cedd-4ea0-b121-f9900c96b0c5"}]': finished 2024-04-14T15:07:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: osdmap e23: 4 total, 3 up, 4 in 2024-04-14T15:07:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:54 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:07:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:55 smithi071 ceph-mon[33150]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1563232974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:55.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:55 smithi045 ceph-mon[29624]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:55.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1563232974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:55.981 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:55 smithi045 ceph-mon[37795]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:55.981 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1563232974' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:07:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:57 smithi071 ceph-mon[33150]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:57.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:57 smithi045 ceph-mon[29624]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:57.866 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:57 smithi045 ceph-mon[37795]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:07:59 smithi071 ceph-mon[33150]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:59.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:07:59 smithi045 ceph-mon[29624]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:07:59.909 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:07:59 smithi045 ceph-mon[37795]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:01.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[29624]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[29624]: Deploying daemon osd.3 on smithi045 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[37795]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:01.801 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:01 smithi045 ceph-mon[37795]: Deploying daemon osd.3 on smithi045 2024-04-14T15:08:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:01 smithi071 ceph-mon[33150]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-14T15:08:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:01 smithi071 ceph-mon[33150]: Deploying daemon osd.3 on smithi045 2024-04-14T15:08:03.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:03 smithi045 ceph-mon[29624]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:03.636 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:03 smithi045 ceph-mon[37795]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:03.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:03 smithi071 ceph-mon[33150]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:05.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:05 smithi071 ceph-mon[33150]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:05.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:05 smithi045 ceph-mon[29624]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:05.910 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:05 smithi045 ceph-mon[37795]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:07.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:07 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:07 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:07 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:07 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:08.141 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:08 smithi045 ceph-mon[37795]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:08.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:08 smithi045 ceph-mon[29624]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:08 smithi071 ceph-mon[33150]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:09.818 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-04-14T15:08:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.817+0000 7f07ddffb700 1 -- 172.21.15.45:0/4215527550 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3088241466) 0x7f07c0000bc0 con 0x7f07cc06f800 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 >> v1:172.21.15.45:6800/2191709230 conn(0x7f07cc06f800 legacy=0x7f07cc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 >> v1:172.21.15.45:6790/0 conn(0x7f07e0105c10 legacy=0x7f07e019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 shutdown_connections 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 >> 172.21.15.45:0/4215527550 conn(0x7f07e00fcd80 msgr2=0x7f07e0108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 shutdown_connections 2024-04-14T15:08:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:08:09.819+0000 7f07e64cc700 1 -- 172.21.15.45:0/4215527550 wait complete. 2024-04-14T15:08:10.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.412 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:10 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.437 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.3.service 2024-04-14T15:08:10.439 INFO:tasks.cephadm:Deploying osd.4 on smithi071 with /dev/vg_nvme/lv_4... 2024-04-14T15:08:10.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-14T15:08:10.469 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:10.469 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.470 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.470 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-14T15:08:10.470 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.470 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:10 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:10.683 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:08:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:11 smithi071 ceph-mon[33150]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-14T15:08:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:11 smithi071 ceph-mon[33150]: osdmap e24: 4 total, 3 up, 4 in 2024-04-14T15:08:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:11 smithi071 ceph-mon[33150]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:08:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:11 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[29624]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-14T15:08:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[29624]: osdmap e24: 4 total, 3 up, 4 in 2024-04-14T15:08:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[29624]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:08:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[37795]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-14T15:08:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[37795]: osdmap e24: 4 total, 3 up, 4 in 2024-04-14T15:08:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[37795]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-14T15:08:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:11.498 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:08:11.203+0000 7fb7e3273700 -1 osd.3 0 waiting for initial osdmap 2024-04-14T15:08:11.498 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:08:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:08:11.208+0000 7fb7df066700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113873 s, 921 MB/s 2024-04-14T15:08:12.478 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-14T15:08:12.478 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:12 smithi071 ceph-mon[33150]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:12.478 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:12 smithi071 ceph-mon[33150]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:08:12.478 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:12 smithi071 ceph-mon[33150]: osdmap e25: 4 total, 3 up, 4 in 2024-04-14T15:08:12.478 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:12.479 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[37795]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[37795]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[37795]: osdmap e25: 4 total, 3 up, 4 in 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[29624]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[29624]: from='osd.3 v1:172.21.15.45:6813/2647051701' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[29624]: osdmap e25: 4 total, 3 up, 4 in 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:12 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:13.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: osd.3 v1:172.21.15.45:6813/2647051701 boot 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: osdmap e26: 4 total, 4 up, 4 in 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:13.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.383 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: osd.3 v1:172.21.15.45:6813/2647051701 boot 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: osdmap e26: 4 total, 4 up, 4 in 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: osd.3 v1:172.21.15.45:6813/2647051701 boot 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: osdmap e26: 4 total, 4 up, 4 in 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:13.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:14.428 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi045 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 4017M 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: osdmap e27: 4 total, 4 up, 4 in 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:14.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:14 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi045 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 4017M 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: osdmap e27: 4 total, 4 up, 4 in 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi045 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 4017M 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: osdmap e27: 4 total, 4 up, 4 in 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:14 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:14.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- 172.21.15.71:0/1222395618 >> v1:172.21.15.71:6789/0 conn(0x7f0434109960 legacy=0x7f043410be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- 172.21.15.71:0/1222395618 shutdown_connections 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- 172.21.15.71:0/1222395618 >> 172.21.15.71:0/1222395618 conn(0x7f04340fcdd0 msgr2=0x7f04340ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- 172.21.15.71:0/1222395618 shutdown_connections 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- 172.21.15.71:0/1222395618 wait complete. 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 Processor -- start 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- start start 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f04341977a0 con 0x7f0434109960 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f04341a2f90 con 0x7f04341017f0 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f043ad3f700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f04341a4170 con 0x7f0434105c20 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f0438adb700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f04341017f0 0x7f0434196c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:60690/0 (socket says 172.21.15.71:60690) 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.919+0000 7f0438adb700 1 -- 172.21.15.71:0/2647060377 learned_addr learned my addr 172.21.15.71:0/2647060377 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133078369 0 0) 0x7f04341a2f90 con 0x7f04341017f0 2024-04-14T15:08:14.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04240035b0 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326967236 0 0) 0x7f04341977a0 con 0x7f0434109960 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04341a2f90 con 0x7f0434109960 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639394614 0 0) 0x7f04341a4170 con 0x7f0434105c20 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04341977a0 con 0x7f0434105c20 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 983566529 0 0) 0x7f04240035b0 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04341a4170 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f04280018f0 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3309586935 0 0) 0x7f04341a4170 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 >> v1:172.21.15.45:6790/0 conn(0x7f0434105c20 legacy=0x7f043419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 >> v1:172.21.15.45:6789/0 conn(0x7f0434109960 legacy=0x7f04341a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04341a5390 con 0x7f04341017f0 2024-04-14T15:08:14.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f04341a1fe0 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0428003b30 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0428005d40 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.920+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f04341a2520 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.921+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04340693d0 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.922+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f04280036a0 con 0x7f04341017f0 2024-04-14T15:08:14.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.922+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(27..27 src has 1..27) ==== 2803+0+0 (unknown 1465224794 0 0) 0x7f0428089550 con 0x7f04341017f0 2024-04-14T15:08:14.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:14.927+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0428057590 con 0x7f04341017f0 2024-04-14T15:08:15.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:15.131+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f04341a2880 con 0x7f04240716d0 2024-04-14T15:08:15.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:15 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:15 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:15 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:15 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:16.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:16 smithi071 ceph-mon[33150]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:16.271 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:16 smithi071 ceph-mon[33150]: from='client.24235 v1:172.21.15.71:0/2647060377' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:16 smithi045 ceph-mon[29624]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:16 smithi045 ceph-mon[29624]: from='client.24235 v1:172.21.15.71:0/2647060377' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:16 smithi045 ceph-mon[37795]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:16 smithi045 ceph-mon[37795]: from='client.24235 v1:172.21.15.71:0/2647060377' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/2877973337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]': finished 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: osdmap e28: 5 total, 4 up, 5 in 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/2164950008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/2877973337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]': finished 2024-04-14T15:08:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: osdmap e28: 5 total, 4 up, 5 in 2024-04-14T15:08:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/2164950008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/2877973337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: from='client.24241 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]: dispatch 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: from='client.24241 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f"}]': finished 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: osdmap e28: 5 total, 4 up, 5 in 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/2164950008' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:20.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:20 smithi071 ceph-mon[33150]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:20.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:20 smithi045 ceph-mon[29624]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:20 smithi045 ceph-mon[37795]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:22 smithi045 ceph-mon[29624]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:22 smithi045 ceph-mon[37795]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:22.518 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:22 smithi071 ceph-mon[33150]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:24.481 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:24 smithi071 ceph-mon[33150]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:24.481 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-14T15:08:24.481 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[29624]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-14T15:08:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[37795]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-14T15:08:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:25.386 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:25 smithi071 ceph-mon[33150]: Deploying daemon osd.4 on smithi071 2024-04-14T15:08:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:25 smithi045 ceph-mon[29624]: Deploying daemon osd.4 on smithi071 2024-04-14T15:08:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:25 smithi045 ceph-mon[37795]: Deploying daemon osd.4 on smithi071 2024-04-14T15:08:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:26 smithi045 ceph-mon[29624]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:26 smithi045 ceph-mon[37795]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:26.595 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:26 smithi071 ceph-mon[33150]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:28 smithi045 ceph-mon[29624]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:28 smithi045 ceph-mon[37795]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:28.527 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:28 smithi071 ceph-mon[33150]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:30.078 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:29 smithi071 ceph-mon[33150]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:30.079 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.079 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.079 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:30.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[29624]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:30.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[37795]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:30.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:32.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:32.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:32 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.447+0000 7f0431ffb700 1 -- 172.21.15.71:0/2647060377 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 3580559844) 0x7f04341a2880 con 0x7f04240716d0 2024-04-14T15:08:32.448 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 4 on host 'smithi071' 2024-04-14T15:08:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.449+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 >> v1:172.21.15.45:6800/2191709230 conn(0x7f04240716d0 legacy=0x7f0424073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.449+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 >> v1:172.21.15.71:6789/0 conn(0x7f04341017f0 legacy=0x7f0434196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.449+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 shutdown_connections 2024-04-14T15:08:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.449+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 >> 172.21.15.71:0/2647060377 conn(0x7f04340fcdd0 msgr2=0x7f043410c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:08:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.450+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 shutdown_connections 2024-04-14T15:08:32.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:32.450+0000 7f043ad3f700 1 -- 172.21.15.71:0/2647060377 wait complete. 2024-04-14T15:08:32.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:32.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-14T15:08:32.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:32.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:32.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:32 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.035 DEBUG:teuthology.orchestra.run.smithi071:osd.4> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.4.service 2024-04-14T15:08:33.038 INFO:tasks.cephadm:Deploying osd.5 on smithi071 with /dev/vg_nvme/lv_3... 2024-04-14T15:08:33.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: osdmap e29: 5 total, 4 up, 5 in 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: osdmap e29: 5 total, 4 up, 5 in 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: osdmap e29: 5 total, 4 up, 5 in 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='osd.4 v1:172.21.15.71:6800/2605280975' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:33.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:33 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:34.107 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:08:33.755+0000 7f2723143700 -1 osd.4 0 waiting for initial osdmap 2024-04-14T15:08:34.107 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:08:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:08:33.761+0000 7f271af2e700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:08:34.216 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:08:34.429 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: osdmap e30: 5 total, 4 up, 5 in 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: from='osd.4 ' entity='osd.4' 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: osd.4 v1:172.21.15.71:6800/2605280975 boot 2024-04-14T15:08:34.430 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:34 smithi071 ceph-mon[33150]: osdmap e31: 5 total, 5 up, 5 in 2024-04-14T15:08:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: osdmap e30: 5 total, 4 up, 5 in 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: from='osd.4 ' entity='osd.4' 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: osd.4 v1:172.21.15.71:6800/2605280975 boot 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[29624]: osdmap e31: 5 total, 5 up, 5 in 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: osdmap e30: 5 total, 4 up, 5 in 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: from='osd.4 ' entity='osd.4' 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: osd.4 v1:172.21.15.71:6800/2605280975 boot 2024-04-14T15:08:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:34 smithi045 ceph-mon[37795]: osdmap e31: 5 total, 5 up, 5 in 2024-04-14T15:08:35.559 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:35 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:08:35.559 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:35 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:08:35.559 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:35 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.559 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:35 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.559 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:35 smithi071 ceph-mon[33150]: osdmap e32: 5 total, 5 up, 5 in 2024-04-14T15:08:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:08:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:08:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[29624]: osdmap e32: 5 total, 5 up, 5 in 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:08:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:35 smithi045 ceph-mon[37795]: osdmap e32: 5 total, 5 up, 5 in 2024-04-14T15:08:36.604 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:36 smithi071 ceph-mon[33150]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:36.604 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:36 smithi071 ceph-mon[33150]: osdmap e33: 5 total, 5 up, 5 in 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124547 s, 842 MB/s 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 2024-04-14T15:08:36.604 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-14T15:08:36.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:36 smithi045 ceph-mon[29624]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:36.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:36 smithi045 ceph-mon[29624]: osdmap e33: 5 total, 5 up, 5 in 2024-04-14T15:08:36.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:36 smithi045 ceph-mon[37795]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-14T15:08:36.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:36 smithi045 ceph-mon[37795]: osdmap e33: 5 total, 5 up, 5 in 2024-04-14T15:08:37.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-04-14T15:08:37.448 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:08:37.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi071 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:37 smithi071 ceph-mon[33150]: osdmap e34: 5 total, 5 up, 5 in 2024-04-14T15:08:37.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi071 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[37795]: osdmap e34: 5 total, 5 up, 5 in 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:37.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi071 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:37.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:37 smithi045 ceph-mon[29624]: osdmap e34: 5 total, 5 up, 5 in 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.073+0000 7f1649bdf700 1 -- 172.21.15.71:0/3887532109 >> v1:172.21.15.71:6789/0 conn(0x7f1644109950 legacy=0x7f164410be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.073+0000 7f1649bdf700 1 -- 172.21.15.71:0/3887532109 shutdown_connections 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.073+0000 7f1649bdf700 1 -- 172.21.15.71:0/3887532109 >> 172.21.15.71:0/3887532109 conn(0x7f16440fc550 msgr2=0x7f16440fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.073+0000 7f1649bdf700 1 -- 172.21.15.71:0/3887532109 shutdown_connections 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.073+0000 7f1649bdf700 1 -- 172.21.15.71:0/3887532109 wait complete. 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1649bdf700 1 Processor -- start 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1649bdf700 1 -- start start 2024-04-14T15:08:39.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1649bdf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f16441018b0 con 0x7f1644102da0 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1649bdf700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f16441a4140 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1649bdf700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f16441a5380 con 0x7f1644109950 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1642ffd700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f1644105d20 0x7f164419e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:34776/0 (socket says 172.21.15.71:34776) 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.074+0000 7f1642ffd700 1 -- 172.21.15.71:0/3644748265 learned_addr learned my addr 172.21.15.71:0/3644748265 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417496212 0 0) 0x7f16441a4140 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16300035b0 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 144410135 0 0) 0x7f16441018b0 con 0x7f1644102da0 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16441a4140 con 0x7f1644102da0 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 298904568 0 0) 0x7f16441a5380 con 0x7f1644109950 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16441018b0 con 0x7f1644109950 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1679320512 0 0) 0x7f16300035b0 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f16441a5380 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f162c001910 con 0x7f1644105d20 2024-04-14T15:08:39.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1320940144 0 0) 0x7f16441a4140 con 0x7f1644102da0 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f16300035b0 con 0x7f1644102da0 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f16340018b0 con 0x7f1644102da0 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2499456845 0 0) 0x7f16441018b0 con 0x7f1644109950 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f16441a4140 con 0x7f1644109950 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f163c001950 con 0x7f1644109950 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 868216766 0 0) 0x7f16441a5380 con 0x7f1644105d20 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 >> v1:172.21.15.45:6790/0 conn(0x7f1644109950 legacy=0x7f16441a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 >> v1:172.21.15.45:6789/0 conn(0x7f1644102da0 legacy=0x7f164419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441a65c0 con 0x7f1644105d20 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f16441a3190 con 0x7f1644105d20 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f16441a36d0 con 0x7f1644105d20 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f162c001d00 con 0x7f1644105d20 2024-04-14T15:08:39.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.075+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f162c005cc0 con 0x7f1644105d20 2024-04-14T15:08:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.076+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1624005150 con 0x7f1644105d20 2024-04-14T15:08:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.076+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f162c01c920 con 0x7f1644105d20 2024-04-14T15:08:39.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.077+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(35..35 src has 1..35) ==== 3205+0+0 (unknown 4034336278 0 0) 0x7f162c089450 con 0x7f1644105d20 2024-04-14T15:08:39.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.081+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f162c057300 con 0x7f1644105d20 2024-04-14T15:08:39.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:38 smithi071 ceph-mon[33150]: osdmap e35: 5 total, 5 up, 5 in 2024-04-14T15:08:39.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:38 smithi045 ceph-mon[29624]: osdmap e35: 5 total, 5 up, 5 in 2024-04-14T15:08:39.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:38 smithi045 ceph-mon[37795]: osdmap e35: 5 total, 5 up, 5 in 2024-04-14T15:08:39.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:39.287+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1624000bc0 con 0x7f163006f9d0 2024-04-14T15:08:40.198 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:39 smithi071 ceph-mon[33150]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-14T15:08:40.199 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:39 smithi071 ceph-mon[33150]: from='client.24262 v1:172.21.15.71:0/3644748265' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:40.199 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:40.199 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:40.199 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:39 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[37795]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[37795]: from='client.24262 v1:172.21.15.71:0/3644748265' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[29624]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[29624]: from='client.24262 v1:172.21.15.71:0/3644748265' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:08:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:39 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/3863752579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]': finished 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: osdmap e36: 6 total, 5 up, 6 in 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/3863752579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]': finished 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: osdmap e36: 6 total, 5 up, 6 in 2024-04-14T15:08:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:41 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/3863752579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: from='client.24268 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]: dispatch 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: from='client.24268 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db3f2e72-cdfb-431f-a7e0-25f137f1d3de"}]': finished 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: osdmap e36: 6 total, 5 up, 6 in 2024-04-14T15:08:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:41 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:43.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/450689104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:43.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/450689104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/450689104' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:08:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:43 smithi045 ceph-mon[29624]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:08:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:43 smithi045 ceph-mon[37795]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:08:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:43 smithi071 ceph-mon[33150]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 140 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:08:46.204 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:45 smithi071 ceph-mon[33150]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-14T15:08:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:45 smithi045 ceph-mon[29624]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-14T15:08:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:45 smithi045 ceph-mon[37795]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-14T15:08:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:47 smithi045 ceph-mon[29624]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-14T15:08:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:47 smithi045 ceph-mon[37795]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-14T15:08:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:47 smithi071 ceph-mon[33150]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-14T15:08:49.061 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-14T15:08:49.061 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:49.061 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:48 smithi071 ceph-mon[33150]: Deploying daemon osd.5 on smithi071 2024-04-14T15:08:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-14T15:08:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[29624]: Deploying daemon osd.5 on smithi071 2024-04-14T15:08:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-14T15:08:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:48 smithi045 ceph-mon[37795]: Deploying daemon osd.5 on smithi071 2024-04-14T15:08:50.226 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:49 smithi071 ceph-mon[33150]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:50.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:49 smithi045 ceph-mon[29624]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:49 smithi045 ceph-mon[37795]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:52.207 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:51 smithi071 ceph-mon[33150]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:51 smithi045 ceph-mon[29624]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:51 smithi045 ceph-mon[37795]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:54 smithi071 ceph-mon[33150]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:54 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:54 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:54 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:54.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[37795]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[29624]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:54 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:56.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[37795]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:56.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:56 smithi045 ceph-mon[29624]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.787 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:56 smithi071 ceph-mon[33150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-14T15:08:56.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.849+0000 7f1640ff9700 1 -- 172.21.15.71:0/3644748265 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 2306797392) 0x7f1624000bc0 con 0x7f163006f9d0 2024-04-14T15:08:56.850 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 5 on host 'smithi071' 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 >> v1:172.21.15.45:6800/2191709230 conn(0x7f163006f9d0 legacy=0x7f1630071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 >> v1:172.21.15.71:6789/0 conn(0x7f1644105d20 legacy=0x7f164419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 shutdown_connections 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 >> 172.21.15.71:0/3644748265 conn(0x7f16440fc550 msgr2=0x7f16440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 shutdown_connections 2024-04-14T15:08:56.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:08:56.851+0000 7f1649bdf700 1 -- 172.21.15.71:0/3644748265 wait complete. 2024-04-14T15:08:57.490 DEBUG:teuthology.orchestra.run.smithi071:osd.5> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.5.service 2024-04-14T15:08:57.492 INFO:tasks.cephadm:Deploying osd.6 on smithi071 with /dev/vg_nvme/lv_2... 2024-04-14T15:08:57.492 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-14T15:08:57.801 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: osdmap e37: 6 total, 5 up, 6 in 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.802 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:57 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-14T15:08:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: osdmap e37: 6 total, 5 up, 6 in 2024-04-14T15:08:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: osdmap e37: 6 total, 5 up, 6 in 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='osd.5 v1:172.21.15.71:6804/3293828982' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:08:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:57 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:08:58.176 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:08:58.825 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:58 smithi071 ceph-mon[33150]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:58.825 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:58 smithi071 ceph-mon[33150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:58.825 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:58 smithi071 ceph-mon[33150]: osdmap e38: 6 total, 5 up, 6 in 2024-04-14T15:08:58.825 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:58.825 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[29624]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[29624]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[29624]: osdmap e38: 6 total, 5 up, 6 in 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[37795]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[37795]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[37795]: osdmap e38: 6 total, 5 up, 6 in 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: osd.5 v1:172.21.15.71:6804/3293828982 boot 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: osdmap e39: 6 total, 6 up, 6 in 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:08:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:08:59 smithi071 ceph-mon[33150]: osdmap e40: 6 total, 6 up, 6 in 2024-04-14T15:08:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:08:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:08:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: osd.5 v1:172.21.15.71:6804/3293828982 boot 2024-04-14T15:08:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: osdmap e39: 6 total, 6 up, 6 in 2024-04-14T15:08:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[29624]: osdmap e40: 6 total, 6 up, 6 in 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: osd.5 v1:172.21.15.71:6804/3293828982 boot 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: osdmap e39: 6 total, 6 up, 6 in 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:08:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:08:59 smithi045 ceph-mon[37795]: osdmap e40: 6 total, 6 up, 6 in 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122248 s, 858 MB/s 2024-04-14T15:09:00.747 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-14T15:09:01.798 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:01 smithi045 ceph-mon[37795]: osdmap e41: 6 total, 6 up, 6 in 2024-04-14T15:09:01.799 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:01 smithi045 ceph-mon[37795]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:01.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:01 smithi045 ceph-mon[29624]: osdmap e41: 6 total, 6 up, 6 in 2024-04-14T15:09:01.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:01 smithi045 ceph-mon[29624]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:01 smithi071 ceph-mon[33150]: osdmap e41: 6 total, 6 up, 6 in 2024-04-14T15:09:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:01 smithi071 ceph-mon[33150]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:01.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-04-14T15:09:02.153 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: osdmap e42: 6 total, 6 up, 6 in 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi071 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.574 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:02.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.820 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: osdmap e42: 6 total, 6 up, 6 in 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi071 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:02.821 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: osdmap e42: 6 total, 6 up, 6 in 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi071 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:02.822 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.750+0000 7f17c3bc1700 1 -- 172.21.15.71:0/1842351328 >> v1:172.21.15.71:6789/0 conn(0x7f17bc109930 legacy=0x7f17bc10bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.750+0000 7f17c3bc1700 1 -- 172.21.15.71:0/1842351328 shutdown_connections 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.750+0000 7f17c3bc1700 1 -- 172.21.15.71:0/1842351328 >> 172.21.15.71:0/1842351328 conn(0x7f17bc0fc0c0 msgr2=0x7f17bc0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.750+0000 7f17c3bc1700 1 -- 172.21.15.71:0/1842351328 shutdown_connections 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.750+0000 7f17c3bc1700 1 -- 172.21.15.71:0/1842351328 wait complete. 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c3bc1700 1 Processor -- start 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c3bc1700 1 -- start start 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c3bc1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f17bc101bf0 con 0x7f17bc109930 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c3bc1700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f17bc1a41a0 con 0x7f17bc102010 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c3bc1700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f17bc1a53c0 con 0x7f17bc105bb0 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c195d700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f17bc102010 0x7f17bc19ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:36242/0 (socket says 172.21.15.71:36242) 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.751+0000 7f17c195d700 1 -- 172.21.15.71:0/3174472183 learned_addr learned my addr 172.21.15.71:0/3174472183 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549986431 0 0) 0x7f17bc1a41a0 con 0x7f17bc102010 2024-04-14T15:09:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17a80035b0 con 0x7f17bc102010 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13248679 0 0) 0x7f17bc101bf0 con 0x7f17bc109930 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17bc1a41a0 con 0x7f17bc109930 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574771971 0 0) 0x7f17bc1a53c0 con 0x7f17bc105bb0 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17bc101bf0 con 0x7f17bc105bb0 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2013037640 0 0) 0x7f17bc1a41a0 con 0x7f17bc109930 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f17bc1a53c0 con 0x7f17bc109930 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f17b40017b0 con 0x7f17bc109930 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1905513565 0 0) 0x7f17a80035b0 con 0x7f17bc102010 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f17bc1a41a0 con 0x7f17bc102010 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1889140706 0 0) 0x7f17bc101bf0 con 0x7f17bc105bb0 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f17a80035b0 con 0x7f17bc105bb0 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f17b0001870 con 0x7f17bc102010 2024-04-14T15:09:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f17b8001770 con 0x7f17bc105bb0 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4107968254 0 0) 0x7f17bc1a53c0 con 0x7f17bc109930 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 >> v1:172.21.15.45:6790/0 conn(0x7f17bc105bb0 legacy=0x7f17bc19e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 >> v1:172.21.15.71:6789/0 conn(0x7f17bc102010 legacy=0x7f17bc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc1a6600 con 0x7f17bc109930 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc1a31f0 con 0x7f17bc109930 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a3730 con 0x7f17bc109930 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f17b4003a00 con 0x7f17bc109930 2024-04-14T15:09:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.752+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f17b40051f0 con 0x7f17bc109930 2024-04-14T15:09:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.753+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a0005150 con 0x7f17bc109930 2024-04-14T15:09:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.754+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f17b4003dd0 con 0x7f17bc109930 2024-04-14T15:09:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.754+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(43..43 src has 1..43) ==== 3497+0+0 (unknown 2795023909 0 0) 0x7f17b40889e0 con 0x7f17bc109930 2024-04-14T15:09:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.759+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f17b4056630 con 0x7f17bc109930 2024-04-14T15:09:03.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:03 smithi071 ceph-mon[33150]: osdmap e43: 6 total, 6 up, 6 in 2024-04-14T15:09:03.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:03 smithi071 ceph-mon[33150]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:03.968+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f17a0000bc0 con 0x7f17a806fa20 2024-04-14T15:09:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:03 smithi045 ceph-mon[37795]: osdmap e43: 6 total, 6 up, 6 in 2024-04-14T15:09:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:03 smithi045 ceph-mon[37795]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:03 smithi045 ceph-mon[29624]: osdmap e43: 6 total, 6 up, 6 in 2024-04-14T15:09:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:03 smithi045 ceph-mon[29624]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:04.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:04 smithi071 ceph-mon[33150]: from='client.14382 v1:172.21.15.71:0/3174472183' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:04.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:04 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:04.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:04 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:04.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:04 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:04.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[29624]: from='client.14382 v1:172.21.15.71:0/3174472183' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:04.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:04.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:04.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:04.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[37795]: from='client.14382 v1:172.21.15.71:0/3174472183' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:04.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:04.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:04.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:04 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:05.640 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:05 smithi071 ceph-mon[33150]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:05 smithi045 ceph-mon[37795]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:05 smithi045 ceph-mon[29624]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: Cluster is now healthy 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1470066243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]': finished 2024-04-14T15:09:06.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: osdmap e44: 7 total, 6 up, 7 in 2024-04-14T15:09:06.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:06 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:06.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-14T15:09:06.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: Cluster is now healthy 2024-04-14T15:09:06.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1470066243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]': finished 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: osdmap e44: 7 total, 6 up, 7 in 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: Cluster is now healthy 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1470066243' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: from='client.24295 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]: dispatch 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: from='client.24295 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2c28b0b-6337-4095-bc52-631380978e4c"}]': finished 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: osdmap e44: 7 total, 6 up, 7 in 2024-04-14T15:09:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:06 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:07.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:07 smithi071 ceph-mon[33150]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:07.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1022792553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:07 smithi045 ceph-mon[29624]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1022792553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:07.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:07 smithi045 ceph-mon[37795]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:07.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1022792553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:09.705 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:09 smithi071 ceph-mon[33150]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-14T15:09:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:09 smithi045 ceph-mon[29624]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-14T15:09:09.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:09 smithi045 ceph-mon[37795]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-14T15:09:11.778 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:11 smithi071 ceph-mon[33150]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:09:11.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:11 smithi045 ceph-mon[29624]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:09:11.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:11 smithi045 ceph-mon[37795]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:09:13.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:13 smithi071 ceph-mon[33150]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:13.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-14T15:09:13.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:13.733 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:13 smithi071 ceph-mon[33150]: Deploying daemon osd.6 on smithi071 2024-04-14T15:09:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[29624]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[29624]: Deploying daemon osd.6 on smithi071 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[37795]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:13 smithi045 ceph-mon[37795]: Deploying daemon osd.6 on smithi071 2024-04-14T15:09:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:15 smithi071 ceph-mon[33150]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:15 smithi045 ceph-mon[29624]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:15 smithi045 ceph-mon[37795]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:17.817 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:17 smithi071 ceph-mon[33150]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:17.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:17 smithi045 ceph-mon[29624]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:17.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:17 smithi045 ceph-mon[37795]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:19.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:19 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:19 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:19 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:19 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:20.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:20 smithi071 ceph-mon[33150]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:20.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:20 smithi045 ceph-mon[29624]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:20 smithi045 ceph-mon[37795]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:21.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.377+0000 7f17aeffd700 1 -- 172.21.15.71:0/3174472183 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 1832967308) 0x7f17a0000bc0 con 0x7f17a806fa20 2024-04-14T15:09:21.378 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 6 on host 'smithi071' 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 >> v1:172.21.15.45:6800/2191709230 conn(0x7f17a806fa20 legacy=0x7f17a8071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 >> v1:172.21.15.45:6789/0 conn(0x7f17bc109930 legacy=0x7f17bc1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 shutdown_connections 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 >> 172.21.15.71:0/3174472183 conn(0x7f17bc0fc0c0 msgr2=0x7f17bc10bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 shutdown_connections 2024-04-14T15:09:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:21.380+0000 7f17c3bc1700 1 -- 172.21.15.71:0/3174472183 wait complete. 2024-04-14T15:09:21.964 DEBUG:teuthology.orchestra.run.smithi071:osd.6> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.6.service 2024-04-14T15:09:21.966 INFO:tasks.cephadm:Deploying osd.7 on smithi071 with /dev/vg_nvme/lv_1... 2024-04-14T15:09:21.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-14T15:09:22.289 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:22.289 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.289 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.290 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.290 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.290 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.290 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:22 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:22 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:22.781 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:09:23.315 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-mon[33150]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-14T15:09:23.316 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-mon[33150]: osdmap e45: 7 total, 6 up, 7 in 2024-04-14T15:09:23.316 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:23.316 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-mon[33150]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:23.316 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-mon[33150]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:23.316 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:09:23.067+0000 7f84df3b3700 -1 osd.6 0 waiting for initial osdmap 2024-04-14T15:09:23.316 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:09:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:09:23.073+0000 7f84db1a6700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:09:23.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[29624]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-14T15:09:23.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[29624]: osdmap e45: 7 total, 6 up, 7 in 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[29624]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[29624]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[37795]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[37795]: osdmap e45: 7 total, 6 up, 7 in 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[37795]: from='osd.6 v1:172.21.15.71:6808/1175230288' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:23 smithi045 ceph-mon[37795]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:24 smithi071 ceph-mon[33150]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:24 smithi071 ceph-mon[33150]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:24 smithi071 ceph-mon[33150]: osdmap e46: 7 total, 6 up, 7 in 2024-04-14T15:09:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[29624]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[29624]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[29624]: osdmap e46: 7 total, 6 up, 7 in 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[37795]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[37795]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[37795]: osdmap e46: 7 total, 6 up, 7 in 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: osd.6 v1:172.21.15.71:6808/1175230288 boot 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: osdmap e47: 7 total, 7 up, 7 in 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:25.185 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:25 smithi071 ceph-mon[33150]: osdmap e48: 7 total, 7 up, 7 in 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122825 s, 854 MB/s 2024-04-14T15:09:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-14T15:09:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:09:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:09:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: osd.6 v1:172.21.15.71:6808/1175230288 boot 2024-04-14T15:09:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: osdmap e47: 7 total, 7 up, 7 in 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[29624]: osdmap e48: 7 total, 7 up, 7 in 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: osd.6 v1:172.21.15.71:6808/1175230288 boot 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: osdmap e47: 7 total, 7 up, 7 in 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:09:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:25 smithi045 ceph-mon[37795]: osdmap e48: 7 total, 7 up, 7 in 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: osdmap e49: 7 total, 7 up, 7 in 2024-04-14T15:09:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:26 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: osdmap e49: 7 total, 7 up, 7 in 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: osdmap e49: 7 total, 7 up, 7 in 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:26.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:26 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:26.655 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:09:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:27 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi071 2024-04-14T15:09:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:27 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-14T15:09:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:27 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi071 2024-04-14T15:09:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:27 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-14T15:09:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:27 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi071 2024-04-14T15:09:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:27 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-14T15:09:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- 172.21.15.71:0/1918760438 >> v1:172.21.15.45:6789/0 conn(0x7f85c8101830 legacy=0x7f85c8101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:28.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- 172.21.15.71:0/1918760438 shutdown_connections 2024-04-14T15:09:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- 172.21.15.71:0/1918760438 >> 172.21.15.71:0/1918760438 conn(0x7f85c80fcdd0 msgr2=0x7f85c80ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- 172.21.15.71:0/1918760438 shutdown_connections 2024-04-14T15:09:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- 172.21.15.71:0/1918760438 wait complete. 2024-04-14T15:09:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 Processor -- start 2024-04-14T15:09:28.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.091+0000 7f85ce853700 1 -- start start 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85ce853700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f85c819af80 con 0x7f85c8101830 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85ce853700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f85c81a63c0 con 0x7f85c8105c60 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85ce853700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f85c81a75a0 con 0x7f85c81099a0 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c77fe700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f85c8105c60 0x7f85c81a0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48934/0 (socket says 172.21.15.71:48934) 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c77fe700 1 -- 172.21.15.71:0/840951525 learned_addr learned my addr 172.21.15.71:0/840951525 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3747460032 0 0) 0x7f85c81a63c0 con 0x7f85c8105c60 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85b00035b0 con 0x7f85c8105c60 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915367828 0 0) 0x7f85c819af80 con 0x7f85c8101830 2024-04-14T15:09:28.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85c81a63c0 con 0x7f85c8101830 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864094572 0 0) 0x7f85c81a75a0 con 0x7f85c81099a0 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85c819af80 con 0x7f85c81099a0 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3501454893 0 0) 0x7f85b00035b0 con 0x7f85c8105c60 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f85c81a75a0 con 0x7f85c8105c60 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85bc0018b0 con 0x7f85c8105c60 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3480092425 0 0) 0x7f85c819af80 con 0x7f85c81099a0 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f85b00035b0 con 0x7f85c81099a0 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.092+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 284812519 0 0) 0x7f85c81a63c0 con 0x7f85c8101830 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f85c819af80 con 0x7f85c8101830 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85c0001c30 con 0x7f85c81099a0 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85b8001430 con 0x7f85c8101830 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1423699038 0 0) 0x7f85c81a75a0 con 0x7f85c8105c60 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 >> v1:172.21.15.45:6790/0 conn(0x7f85c81099a0 legacy=0x7f85c81a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 >> v1:172.21.15.45:6789/0 conn(0x7f85c8101830 legacy=0x7f85c819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:28.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c81a87c0 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f85c81a5410 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f85c81a5950 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f85bc003b30 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.093+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85bc005ce0 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.094+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b4005150 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.094+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f85bc01c6c0 con 0x7f85c8105c60 2024-04-14T15:09:28.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.095+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(51..51 src has 1..51) ==== 3789+0+0 (unknown 506872933 0 0) 0x7f85bc089790 con 0x7f85c8105c60 2024-04-14T15:09:28.099 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.099+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f85bc057320 con 0x7f85c8105c60 2024-04-14T15:09:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:28.303+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f85b4000bc0 con 0x7f85b006f910 2024-04-14T15:09:28.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:28 smithi071 ceph-mon[33150]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:28 smithi071 ceph-mon[33150]: osdmap e50: 7 total, 7 up, 7 in 2024-04-14T15:09:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:28 smithi045 ceph-mon[29624]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:28 smithi045 ceph-mon[29624]: osdmap e50: 7 total, 7 up, 7 in 2024-04-14T15:09:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:28 smithi045 ceph-mon[37795]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:28 smithi045 ceph-mon[37795]: osdmap e50: 7 total, 7 up, 7 in 2024-04-14T15:09:29.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:29 smithi071 ceph-mon[33150]: osdmap e51: 7 total, 7 up, 7 in 2024-04-14T15:09:29.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:29 smithi071 ceph-mon[33150]: from='client.24313 v1:172.21.15.71:0/840951525' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:29.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:29.215 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:29.216 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:29 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:29.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[29624]: osdmap e51: 7 total, 7 up, 7 in 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[29624]: from='client.24313 v1:172.21.15.71:0/840951525' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[37795]: osdmap e51: 7 total, 7 up, 7 in 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[37795]: from='client.24313 v1:172.21.15.71:0/840951525' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-14T15:09:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:29 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:30.247 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:30 smithi071 ceph-mon[33150]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:30 smithi045 ceph-mon[37795]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:30 smithi045 ceph-mon[29624]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1757597881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:31 smithi071 ceph-mon[33150]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:31 smithi071 ceph-mon[33150]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]': finished 2024-04-14T15:09:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:31 smithi071 ceph-mon[33150]: osdmap e52: 8 total, 7 up, 8 in 2024-04-14T15:09:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:31 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1757597881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[37795]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[37795]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]': finished 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[37795]: osdmap e52: 8 total, 7 up, 8 in 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1757597881' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[29624]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]: dispatch 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[29624]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1374c765-77e1-49d7-88b5-cbb13810ef94"}]': finished 2024-04-14T15:09:31.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[29624]: osdmap e52: 8 total, 7 up, 8 in 2024-04-14T15:09:31.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:31 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:32 smithi071 ceph-mon[33150]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:09:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/2438115142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:32 smithi045 ceph-mon[29624]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:09:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/2438115142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:32 smithi045 ceph-mon[37795]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-14T15:09:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/2438115142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-14T15:09:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:34 smithi071 ceph-mon[33150]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:34 smithi045 ceph-mon[29624]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:34 smithi045 ceph-mon[37795]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:36.193 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:36 smithi071 ceph-mon[33150]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-14T15:09:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:36 smithi045 ceph-mon[29624]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-14T15:09:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:36 smithi045 ceph-mon[37795]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-14T15:09:38.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:38 smithi071 ceph-mon[33150]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-14T15:09:38.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-14T15:09:38.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:38 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[29624]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[37795]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-14T15:09:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:38 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:39 smithi071 ceph-mon[33150]: Deploying daemon osd.7 on smithi071 2024-04-14T15:09:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:39 smithi045 ceph-mon[37795]: Deploying daemon osd.7 on smithi071 2024-04-14T15:09:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:39 smithi045 ceph-mon[29624]: Deploying daemon osd.7 on smithi071 2024-04-14T15:09:40.341 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:40 smithi071 ceph-mon[33150]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:40 smithi045 ceph-mon[29624]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:40 smithi045 ceph-mon[37795]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-14T15:09:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:42 smithi071 ceph-mon[33150]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:42 smithi045 ceph-mon[29624]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:42 smithi045 ceph-mon[37795]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:43 smithi071 ceph-mon[33150]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:43 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:43 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:43 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[29624]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[37795]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:43 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:45 smithi071 ceph-mon[33150]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:45 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:45 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[37795]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[29624]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:45 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:46.569 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 7 on host 'smithi071' 2024-04-14T15:09:46.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.568+0000 7f85c57fa700 1 -- 172.21.15.71:0/840951525 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (unknown 0 0 827667000) 0x7f85b4000bc0 con 0x7f85b006f910 2024-04-14T15:09:46.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 >> v1:172.21.15.45:6800/2191709230 conn(0x7f85b006f910 legacy=0x7f85b0071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:46.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 >> v1:172.21.15.71:6789/0 conn(0x7f85c8105c60 legacy=0x7f85c81a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:46.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 shutdown_connections 2024-04-14T15:09:46.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 >> 172.21.15.71:0/840951525 conn(0x7f85c80fcdd0 msgr2=0x7f85c806adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:46.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 shutdown_connections 2024-04-14T15:09:46.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:46.570+0000 7f85ce853700 1 -- 172.21.15.71:0/840951525 wait complete. 2024-04-14T15:09:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:46 smithi071 ceph-mon[33150]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:46 smithi071 ceph-mon[33150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:46 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:46 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.240 DEBUG:teuthology.orchestra.run.smithi071:osd.7> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.7.service 2024-04-14T15:09:47.242 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-04-14T15:09:47.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd stat -f json 2024-04-14T15:09:47.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[29624]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[29624]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[37795]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[37795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-14T15:09:47.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:46 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:47.500 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:09:47.917 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:47.917 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-14T15:09:47.917 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: osdmap e53: 8 total, 7 up, 8 in 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[37795]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: osdmap e53: 8 total, 7 up, 8 in 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:47 smithi045 ceph-mon[29624]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:48.252 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:48.252 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-14T15:09:48.252 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: osdmap e53: 8 total, 7 up, 8 in 2024-04-14T15:09:48.253 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: from='osd.7 v1:172.21.15.71:6812/2779299537' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:48.253 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:48.253 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:47 smithi071 ceph-mon[33150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-14T15:09:48.568 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:09:48.293+0000 7fd30fb79700 -1 osd.7 0 waiting for initial osdmap 2024-04-14T15:09:48.568 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:09:48.299+0000 7fd307964700 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-14T15:09:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.104+0000 7f0124f03700 1 -- 172.21.15.45:0/288237374 >> v1:172.21.15.45:6790/0 conn(0x7f01201017e0 legacy=0x7f0120101c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.104+0000 7f0124f03700 1 -- 172.21.15.45:0/288237374 shutdown_connections 2024-04-14T15:09:49.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.104+0000 7f0124f03700 1 -- 172.21.15.45:0/288237374 >> 172.21.15.45:0/288237374 conn(0x7f01200fcd80 msgr2=0x7f01200ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.104+0000 7f0124f03700 1 -- 172.21.15.45:0/288237374 shutdown_connections 2024-04-14T15:09:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.104+0000 7f0124f03700 1 -- 172.21.15.45:0/288237374 wait complete. 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f0124f03700 1 Processor -- start 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f0124f03700 1 -- start start 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f0124f03700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f012006bc10 con 0x7f0120109950 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f0124f03700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01201a3f40 con 0x7f0120105c10 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f0124f03700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f01201a5180 con 0x7f01201017e0 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f011e59c700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f01201017e0 0x7f012019a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53016/0 (socket says 172.21.15.45:53016) 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f011ed9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0120109950 0x7f01201a17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57634/0 (socket says 172.21.15.45:57634) 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f011e59c700 1 -- 172.21.15.45:0/202539868 learned_addr learned my addr 172.21.15.45:0/202539868 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:09:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2501977639 0 0) 0x7f01201a5180 con 0x7f01201017e0 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f010c005240 con 0x7f01201017e0 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399927627 0 0) 0x7f012006bc10 con 0x7f0120109950 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01201a5180 con 0x7f0120109950 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537243435 0 0) 0x7f01201a3f40 con 0x7f0120105c10 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.105+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f012006bc10 con 0x7f0120105c10 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1979969272 0 0) 0x7f012006bc10 con 0x7f0120105c10 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01201a3f40 con 0x7f0120105c10 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0108001230 con 0x7f0120105c10 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2089202733 0 0) 0x7f010c005240 con 0x7f01201017e0 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f012006bc10 con 0x7f01201017e0 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 122575435 0 0) 0x7f01201a5180 con 0x7f0120109950 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f010c005240 con 0x7f0120109950 2024-04-14T15:09:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0118001050 con 0x7f01201017e0 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0110001bb0 con 0x7f0120109950 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3800924493 0 0) 0x7f01201a3f40 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 >> v1:172.21.15.45:6790/0 conn(0x7f01201017e0 legacy=0x7f012019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 >> v1:172.21.15.45:6789/0 conn(0x7f0120109950 legacy=0x7f01201a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01201a63c0 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f01201a53b0 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f01201a58f0 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0108003b10 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.106+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f01080049a0 con 0x7f0120105c10 2024-04-14T15:09:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.107+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0100005150 con 0x7f0120105c10 2024-04-14T15:09:49.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.107+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f0108003680 con 0x7f0120105c10 2024-04-14T15:09:49.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.108+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(55..55 src has 1..55) ==== 4114+0+0 (unknown 4275921239 0 0) 0x7f01080885c0 con 0x7f0120105c10 2024-04-14T15:09:49.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.113+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0108055fa0 con 0x7f0120105c10 2024-04-14T15:09:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[29624]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[29624]: osdmap e54: 8 total, 7 up, 8 in 2024-04-14T15:09:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[29624]: from='osd.7 ' entity='osd.7' 2024-04-14T15:09:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[37795]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[37795]: osdmap e54: 8 total, 7 up, 8 in 2024-04-14T15:09:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:48 smithi045 ceph-mon[37795]: from='osd.7 ' entity='osd.7' 2024-04-14T15:09:49.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-mon[33150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-14T15:09:49.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-mon[33150]: osdmap e54: 8 total, 7 up, 8 in 2024-04-14T15:09:49.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:49.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:48 smithi071 ceph-mon[33150]: from='osd.7 ' entity='osd.7' 2024-04-14T15:09:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.330+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0100005440 con 0x7f0120105c10 2024-04-14T15:09:49.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.330+0000 7f01177fe700 1 -- 172.21.15.45:0/202539868 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v55) ==== 74+0+130 (unknown 171817225 0 2502700304) 0x7f01080558a0 con 0x7f0120105c10 2024-04-14T15:09:49.330 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:09:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 >> v1:172.21.15.45:6800/2191709230 conn(0x7f010c06f9a0 legacy=0x7f010c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:49.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 >> v1:172.21.15.71:6789/0 conn(0x7f0120105c10 legacy=0x7f012019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 shutdown_connections 2024-04-14T15:09:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 >> 172.21.15.45:0/202539868 conn(0x7f01200fcd80 msgr2=0x7f0120108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 shutdown_connections 2024-04-14T15:09:49.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:49.332+0000 7f0124f03700 1 -- 172.21.15.45:0/202539868 wait complete. 2024-04-14T15:09:49.958 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1713107388,"num_in_osds":8,"osd_in_since":1713107370,"num_remapped_pgs":1} 2024-04-14T15:09:49.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd dump --format=json 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: purged_snaps scrub starts 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: purged_snaps scrub ok 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: osd.7 v1:172.21.15.71:6812/2779299537 boot 2024-04-14T15:09:50.002 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: osdmap e55: 8 total, 8 up, 8 in 2024-04-14T15:09:50.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.003 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/202539868' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-14T15:09:50.219 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: purged_snaps scrub starts 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: purged_snaps scrub ok 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: osd.7 v1:172.21.15.71:6812/2779299537 boot 2024-04-14T15:09:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: osdmap e55: 8 total, 8 up, 8 in 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/202539868' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: purged_snaps scrub starts 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: purged_snaps scrub ok 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: osd.7 v1:172.21.15.71:6812/2779299537 boot 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: osdmap e55: 8 total, 8 up, 8 in 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:09:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/202539868' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: osdmap e56: 8 total, 8 up, 8 in 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: Detected new or changed devices on smithi071 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: osdmap e56: 8 total, 8 up, 8 in 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: Detected new or changed devices on smithi071 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:50.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:50 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.237 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: osdmap e56: 8 total, 8 up, 8 in 2024-04-14T15:09:51.237 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: Detected new or changed devices on smithi071 2024-04-14T15:09:51.237 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:51.238 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:51.239 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:50 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.856+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2422664018 >> v1:172.21.15.45:6789/0 conn(0x7f5a741077a0 legacy=0x7f5a74107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.856+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2422664018 shutdown_connections 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.856+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2422664018 >> 172.21.15.45:0/2422664018 conn(0x7f5a740fd2b0 msgr2=0x7f5a740ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.856+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2422664018 shutdown_connections 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.856+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2422664018 wait complete. 2024-04-14T15:09:51.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a7a76d700 1 Processor -- start 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a7a76d700 1 -- start start 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a7a76d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a741a9c70 con 0x7f5a741077a0 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a7a76d700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a741aae50 con 0x7f5a7410e220 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a7a76d700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f5a741ac090 con 0x7f5a7410a650 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a737fe700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f5a7410a650 0x7f5a741a4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44374/0 (socket says 172.21.15.45:44374) 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a737fe700 1 -- 172.21.15.45:0/2963512702 learned_addr learned my addr 172.21.15.45:0/2963512702 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:09:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a73fff700 1 --1- 172.21.15.45:0/2963512702 >> v1:172.21.15.45:6789/0 conn(0x7f5a741077a0 0x7f5a7410da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45840/0 (socket says 172.21.15.45:45840) 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403643182 0 0) 0x7f5a741a9c70 con 0x7f5a741077a0 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.857+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a600031f0 con 0x7f5a741077a0 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567166647 0 0) 0x7f5a741ac090 con 0x7f5a7410a650 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a741a9c70 con 0x7f5a7410a650 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2603981984 0 0) 0x7f5a741aae50 con 0x7f5a7410e220 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a741ac090 con 0x7f5a7410e220 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 121783319 0 0) 0x7f5a600031f0 con 0x7f5a741077a0 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a741aae50 con 0x7f5a741077a0 2024-04-14T15:09:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3221358973 0 0) 0x7f5a741a9c70 con 0x7f5a7410a650 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a600031f0 con 0x7f5a7410a650 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 519578456 0 0) 0x7f5a741ac090 con 0x7f5a7410e220 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a741a9c70 con 0x7f5a7410e220 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f5a5c0042e0 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f5a640017f0 con 0x7f5a7410a650 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f5a6c001c10 con 0x7f5a7410e220 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 565470393 0 0) 0x7f5a741aae50 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 >> v1:172.21.15.45:6790/0 conn(0x7f5a7410a650 legacy=0x7f5a741a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 >> v1:172.21.15.71:6789/0 conn(0x7f5a7410e220 legacy=0x7f5a741a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741ad2d0 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741ac2c0 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f5a741ac780 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f5a5c001ae0 con 0x7f5a741077a0 2024-04-14T15:09:51.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.858+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f5a5c004de0 con 0x7f5a741077a0 2024-04-14T15:09:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.859+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a54005120 con 0x7f5a741077a0 2024-04-14T15:09:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.860+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f5a5c004dc0 con 0x7f5a741077a0 2024-04-14T15:09:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.860+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(57..57 src has 1..57) ==== 4110+0+0 (unknown 1141179619 0 0) 0x7f5a5c0899f0 con 0x7f5a741077a0 2024-04-14T15:09:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:51.865+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f5a5c057420 con 0x7f5a741077a0 2024-04-14T15:09:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:51 smithi045 ceph-mon[29624]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:51 smithi045 ceph-mon[29624]: osdmap e57: 8 total, 8 up, 8 in 2024-04-14T15:09:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:51 smithi045 ceph-mon[37795]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:51 smithi045 ceph-mon[37795]: osdmap e57: 8 total, 8 up, 8 in 2024-04-14T15:09:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.075+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a54005410 con 0x7f5a741077a0 2024-04-14T15:09:52.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.075+0000 7f5a717fa700 1 -- 172.21.15.45:0/2963512702 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+11473 (unknown 2626357731 0 117133348) 0x7f5a5c056d20 con 0x7f5a741077a0 2024-04-14T15:09:52.076 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:09:52.076 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":58,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","created":"2024-04-14T15:04:40.069266+0000","modified":"2024-04-14T15:09:51.876125+0000","last_up_change":"2024-04-14T15:09:48.865155+0000","last_in_change":"2024-04-14T15:09:30.892466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T15:07:48.583816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ceb62cce-7685-4b96-930f-bba7be80eabd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":609845118}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":609845118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":609845118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":609845118}]},"public_addr":"172.21.15.45:6801/609845118","cluster_addr":"172.21.15.45:6802/609845118","heartbeat_back_addr":"172.21.15.45:6804/609845118","heartbeat_front_addr":"172.21.15.45:6803/609845118","state":["exists","up"]},{"osd":1,"uuid":"39c5f730-a033-4475-9c37-37fe6929650d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":1698650141}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":1698650141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":1698650141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":1698650141}]},"public_addr":"172.21.15.45:6805/1698650141","cluster_addr":"172.21.15.45:6806/1698650141","heartbeat_back_addr":"172.21.15.45:6808/1698650141","heartbeat_front_addr":"172.21.15.45:6807/1698650141","state":["exists","up"]},{"osd":2,"uuid":"11bba3d9-26dc-4989-ae9d-555847390c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":1422326616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":1422326616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6812","nonce":1422326616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":1422326616}]},"public_addr":"172.21.15.45:6809/1422326616","cluster_addr":"172.21.15.45:6810/1422326616","heartbeat_back_addr":"172.21.15.45:6812/1422326616","heartbeat_front_addr":"172.21.15.45:6811/1422326616","state":["exists","up"]},{"osd":3,"uuid":"16ce0a73-cedd-4ea0-b121-f9900c96b0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6813","nonce":2647051701}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6814","nonce":2647051701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6816","nonce":2647051701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6815","nonce":2647051701}]},"public_addr":"172.21.15.45:6813/2647051701","cluster_addr":"172.21.15.45:6814/2647051701","heartbeat_back_addr":"172.21.15.45:6816/2647051701","heartbeat_front_addr":"172.21.15.45:6815/2647051701","state":["exists","up"]},{"osd":4,"uuid":"35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6800","nonce":2605280975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":2605280975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":2605280975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":2605280975}]},"public_addr":"172.21.15.71:6800/2605280975","cluster_addr":"172.21.15.71:6801/2605280975","heartbeat_back_addr":"172.21.15.71:6803/2605280975","heartbeat_front_addr":"172.21.15.71:6802/2605280975","state":["exists","up"]},{"osd":5,"uuid":"db3f2e72-cdfb-431f-a7e0-25f137f1d3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":3293828982}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":3293828982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":3293828982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":3293828982}]},"public_addr":"172.21.15.71:6804/3293828982","cluster_addr":"172.21.15.71:6805/3293828982","heartbeat_back_addr":"172.21.15.71:6807/3293828982","heartbeat_front_addr":"172.21.15.71:6806/3293828982","state":["exists","up"]},{"osd":6,"uuid":"d2c28b0b-6337-4095-bc52-631380978e4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":1175230288}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":1175230288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":1175230288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":1175230288}]},"public_addr":"172.21.15.71:6808/1175230288","cluster_addr":"172.21.15.71:6809/1175230288","heartbeat_back_addr":"172.21.15.71:6811/1175230288","heartbeat_front_addr":"172.21.15.71:6810/1175230288","state":["exists","up"]},{"osd":7,"uuid":"1374c765-77e1-49d7-88b5-cbb13810ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":2779299537}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":2779299537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":2779299537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":2779299537}]},"public_addr":"172.21.15.71:6812/2779299537","cluster_addr":"172.21.15.71:6813/2779299537","heartbeat_back_addr":"172.21.15.71:6815/2779299537","heartbeat_front_addr":"172.21.15.71:6814/2779299537","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:06:56.929871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:20.714342+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:45.708756+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:10.267994+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:32.921703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:57.376518+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:22.153259+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:46.961704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4130748887":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/2960091533":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/3894205139":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/2089996418":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/3535859446":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/2598686815":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/3911040716":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/1260581702":"2024-04-15T15:05:38.510265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.077+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 >> v1:172.21.15.45:6800/2191709230 conn(0x7f5a6006fa40 legacy=0x7f5a60071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.077+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 >> v1:172.21.15.45:6789/0 conn(0x7f5a741077a0 legacy=0x7f5a7410da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.078+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 shutdown_connections 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.078+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 >> 172.21.15.45:0/2963512702 conn(0x7f5a740fd2b0 msgr2=0x7f5a74110eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.078+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 shutdown_connections 2024-04-14T15:09:52.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:52.078+0000 7f5a7a76d700 1 -- 172.21.15.45:0/2963512702 wait complete. 2024-04-14T15:09:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:51 smithi071 ceph-mon[33150]: pgmap v172: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:51 smithi071 ceph-mon[33150]: osdmap e57: 8 total, 8 up, 8 in 2024-04-14T15:09:52.711 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-14T15:07:48.583816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-14T15:09:52.711 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-14T15:09:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:09:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:52 smithi045 ceph-mon[29624]: osdmap e58: 8 total, 8 up, 8 in 2024-04-14T15:09:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2963512702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:09:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:52 smithi045 ceph-mon[37795]: osdmap e58: 8 total, 8 up, 8 in 2024-04-14T15:09:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2963512702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:09:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:52 smithi071 ceph-mon[33150]: osdmap e58: 8 total, 8 up, 8 in 2024-04-14T15:09:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2963512702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:09:54.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:53 smithi045 ceph-mon[37795]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:54.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:53 smithi045 ceph-mon[37795]: osdmap e59: 8 total, 8 up, 8 in 2024-04-14T15:09:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:53 smithi045 ceph-mon[29624]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:53 smithi045 ceph-mon[29624]: osdmap e59: 8 total, 8 up, 8 in 2024-04-14T15:09:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:53 smithi071 ceph-mon[33150]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:53 smithi071 ceph-mon[33150]: osdmap e59: 8 total, 8 up, 8 in 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.494+0000 7fa282455700 1 -- 172.21.15.45:0/2453330735 >> v1:172.21.15.45:6790/0 conn(0x7fa27c103a60 legacy=0x7fa27c103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.494+0000 7fa282455700 1 -- 172.21.15.45:0/2453330735 shutdown_connections 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.494+0000 7fa282455700 1 -- 172.21.15.45:0/2453330735 >> 172.21.15.45:0/2453330735 conn(0x7fa27c0fd2b0 msgr2=0x7fa27c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.494+0000 7fa282455700 1 -- 172.21.15.45:0/2453330735 shutdown_connections 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.494+0000 7fa282455700 1 -- 172.21.15.45:0/2453330735 wait complete. 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa282455700 1 Processor -- start 2024-04-14T15:09:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa282455700 1 -- start start 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa282455700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa27c1a7810 con 0x7fa27c103a60 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa282455700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa27c1a8a30 con 0x7fa27c10b870 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa282455700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa27c1a9c70 con 0x7fa27c10f380 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa27bfff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa27c103a60 0x7fa27c10af30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45850/0 (socket says 172.21.15.45:45850) 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa27bfff700 1 -- 172.21.15.45:0/2012236743 learned_addr learned my addr 172.21.15.45:0/2012236743 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.495+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121310078 0 0) 0x7fa27c1a7810 con 0x7fa27c103a60 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2680031f0 con 0x7fa27c103a60 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951463732 0 0) 0x7fa27c1a9c70 con 0x7fa27c10f380 2024-04-14T15:09:54.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa27c1a7810 con 0x7fa27c10f380 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2672307593 0 0) 0x7fa27c1a8a30 con 0x7fa27c10b870 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa27c1a9c70 con 0x7fa27c10b870 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3657296130 0 0) 0x7fa2680031f0 con 0x7fa27c103a60 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa27c1a8a30 con 0x7fa27c103a60 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa26c004540 con 0x7fa27c103a60 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4195372619 0 0) 0x7fa27c1a7810 con 0x7fa27c10f380 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa2680031f0 con 0x7fa27c10f380 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa274001b70 con 0x7fa27c10f380 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1869272640 0 0) 0x7fa27c1a9c70 con 0x7fa27c10b870 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa27c1a7810 con 0x7fa27c10b870 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa264001890 con 0x7fa27c10b870 2024-04-14T15:09:54.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3323689817 0 0) 0x7fa27c1a8a30 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 >> v1:172.21.15.45:6790/0 conn(0x7fa27c10f380 legacy=0x7fa27c1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 >> v1:172.21.15.71:6789/0 conn(0x7fa27c10b870 legacy=0x7fa27c1a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c1aaeb0 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c1a9ea0 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa27c1aa360 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fa26c001be0 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.496+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa26c004eb0 con 0x7fa27c103a60 2024-04-14T15:09:54.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.497+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c005150 con 0x7fa27c103a60 2024-04-14T15:09:54.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.498+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7fa26c004eb0 con 0x7fa27c103a60 2024-04-14T15:09:54.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.498+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 1303600178 0 0) 0x7fa26c089a50 con 0x7fa27c103a60 2024-04-14T15:09:54.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.503+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fa26c0574b0 con 0x7fa27c103a60 2024-04-14T15:09:54.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.709+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa25c005c50 con 0x7fa27c103a60 2024-04-14T15:09:54.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.709+0000 7fa2797fa700 1 -- 172.21.15.45:0/2012236743 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (unknown 4013917427 0 2170607528) 0x7fa26c056db0 con 0x7fa27c103a60 2024-04-14T15:09:54.710 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.711+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 >> v1:172.21.15.45:6800/2191709230 conn(0x7fa26806f9a0 legacy=0x7fa268071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.711+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 >> v1:172.21.15.45:6789/0 conn(0x7fa27c103a60 legacy=0x7fa27c10af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.711+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 shutdown_connections 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.711+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 >> 172.21.15.45:0/2012236743 conn(0x7fa27c0fd2b0 msgr2=0x7fa27c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.712+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 shutdown_connections 2024-04-14T15:09:54.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:09:54.712+0000 7fa282455700 1 -- 172.21.15.45:0/2012236743 wait complete. 2024-04-14T15:09:54.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2012236743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-14T15:09:54.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2012236743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-14T15:09:55.251 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi045 2024-04-14T15:09:55.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi045=foo.a' 2024-04-14T15:09:55.282 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2012236743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-14T15:09:55.505 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:09:56.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:55 smithi045 ceph-mon[29624]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:55 smithi045 ceph-mon[37795]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:56.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:55 smithi071 ceph-mon[33150]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:09:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.041+0000 7fc0491a8700 1 -- 172.21.15.71:0/158583071 >> v1:172.21.15.71:6789/0 conn(0x7fc044105c20 legacy=0x7fc044108070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.041+0000 7fc0491a8700 1 -- 172.21.15.71:0/158583071 shutdown_connections 2024-04-14T15:09:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.041+0000 7fc0491a8700 1 -- 172.21.15.71:0/158583071 >> 172.21.15.71:0/158583071 conn(0x7fc0440fcd90 msgr2=0x7fc0440ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.042+0000 7fc0491a8700 1 -- 172.21.15.71:0/158583071 shutdown_connections 2024-04-14T15:09:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.042+0000 7fc0491a8700 1 -- 172.21.15.71:0/158583071 wait complete. 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.042+0000 7fc0491a8700 1 Processor -- start 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.042+0000 7fc0491a8700 1 -- start start 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc0491a8700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc04406bd50 con 0x7fc044109960 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc0491a8700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc0441a4180 con 0x7fc0441017f0 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc0491a8700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fc0441a53c0 con 0x7fc044105c20 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc042d9d700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fc0441017f0 0x7fc04419abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:47056/0 (socket says 172.21.15.71:47056) 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc042d9d700 1 -- 172.21.15.71:0/2032426174 learned_addr learned my addr 172.21.15.71:0/2032426174 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721581538 0 0) 0x7fc0441a4180 con 0x7fc0441017f0 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc02c0035b0 con 0x7fc0441017f0 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947238997 0 0) 0x7fc04406bd50 con 0x7fc044109960 2024-04-14T15:09:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0441a4180 con 0x7fc044109960 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538198982 0 0) 0x7fc0441a53c0 con 0x7fc044105c20 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc04406bd50 con 0x7fc044105c20 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2301812307 0 0) 0x7fc02c0035b0 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc0441a53c0 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fc034001870 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1693339571 0 0) 0x7fc0441a4180 con 0x7fc044109960 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc02c0035b0 con 0x7fc044109960 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fc03c001b70 con 0x7fc044109960 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2052929723 0 0) 0x7fc04406bd50 con 0x7fc044105c20 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc0441a4180 con 0x7fc044105c20 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fc038001430 con 0x7fc044105c20 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2561580340 0 0) 0x7fc0441a53c0 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 >> v1:172.21.15.45:6790/0 conn(0x7fc044105c20 legacy=0x7fc04419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 >> v1:172.21.15.45:6789/0 conn(0x7fc044109960 legacy=0x7fc0441a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.043+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0441a6600 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.044+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441a55f0 con 0x7fc0441017f0 2024-04-14T15:09:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.044+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fc0441a5b30 con 0x7fc0441017f0 2024-04-14T15:09:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.044+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fc034001c40 con 0x7fc0441017f0 2024-04-14T15:09:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.044+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fc034005d00 con 0x7fc0441017f0 2024-04-14T15:09:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.045+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030005150 con 0x7fc0441017f0 2024-04-14T15:09:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.045+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7fc03401c960 con 0x7fc0441017f0 2024-04-14T15:09:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.045+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 1303600178 0 0) 0x7fc0340898e0 con 0x7fc0441017f0 2024-04-14T15:09:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.050+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fc034057420 con 0x7fc0441017f0 2024-04-14T15:09:57.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.256+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi045=foo.a", "target": ["mon-mgr", ""]}) -- 0x7fc030000bc0 con 0x7fc02c06f8c0 2024-04-14T15:09:57.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.263+0000 7fc02bfff700 1 -- 172.21.15.71:0/2032426174 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (unknown 0 0 1123153589) 0x7fc030000bc0 con 0x7fc02c06f8c0 2024-04-14T15:09:57.263 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled rgw.foo.a update... 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 >> v1:172.21.15.45:6800/2191709230 conn(0x7fc02c06f8c0 legacy=0x7fc02c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 >> v1:172.21.15.71:6789/0 conn(0x7fc0441017f0 legacy=0x7fc04419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 shutdown_connections 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 >> 172.21.15.71:0/2032426174 conn(0x7fc0440fcd90 msgr2=0x7fc044108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 shutdown_connections 2024-04-14T15:09:57.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:57.265+0000 7fc0491a8700 1 -- 172.21.15.71:0/2032426174 wait complete. 2024-04-14T15:09:57.876 DEBUG:teuthology.orchestra.run.smithi045:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@rgw.foo.a.service 2024-04-14T15:09:57.878 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi071 2024-04-14T15:09:57.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-04-14T15:09:58.149 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:09:58.520 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='client.24355 v1:172.21.15.71:0/2032426174' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi045=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.521 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:09:58 smithi071 ceph-mon[33150]: Deploying daemon rgw.foo.a on smithi045 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='client.24355 v1:172.21.15.71:0/2032426174' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi045=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[29624]: Deploying daemon rgw.foo.a on smithi045 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='client.24355 v1:172.21.15.71:0/2032426174' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi045=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-04-14T15:09:58.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:09:58.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:09:58.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:09:58 smithi045 ceph-mon[37795]: Deploying daemon rgw.foo.a on smithi045 2024-04-14T15:09:59.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.725+0000 7fd8c8c67700 1 -- 172.21.15.71:0/3465543863 >> v1:172.21.15.71:6789/0 conn(0x7fd8c4106d20 legacy=0x7fd8c4109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.725+0000 7fd8c8c67700 1 -- 172.21.15.71:0/3465543863 shutdown_connections 2024-04-14T15:09:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.725+0000 7fd8c8c67700 1 -- 172.21.15.71:0/3465543863 >> 172.21.15.71:0/3465543863 conn(0x7fd8c40fdeb0 msgr2=0x7fd8c41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:09:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.725+0000 7fd8c8c67700 1 -- 172.21.15.71:0/3465543863 shutdown_connections 2024-04-14T15:09:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.725+0000 7fd8c8c67700 1 -- 172.21.15.71:0/3465543863 wait complete. 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c8c67700 1 Processor -- start 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c8c67700 1 -- start start 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c8c67700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8c4198a10 con 0x7fd8c41028f0 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c8c67700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8c41a4200 con 0x7fd8c410aa60 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c8c67700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8c41a53e0 con 0x7fd8c4106d20 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c2d9d700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fd8c410aa60 0x7fd8c41a1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:47072/0 (socket says 172.21.15.71:47072) 2024-04-14T15:09:59.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c259c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8c41028f0 0x7fd8c4197ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.71:41336/0 (socket says 172.21.15.71:41336) 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8c2d9d700 1 -- 172.21.15.71:0/626713869 learned_addr learned my addr 172.21.15.71:0/626713869 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.726+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2189159817 0 0) 0x7fd8c41a4200 con 0x7fd8c410aa60 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8ac0035b0 con 0x7fd8c410aa60 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847578734 0 0) 0x7fd8c4198a10 con 0x7fd8c41028f0 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8c41a4200 con 0x7fd8c41028f0 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744586614 0 0) 0x7fd8c41a53e0 con 0x7fd8c4106d20 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8c4198a10 con 0x7fd8c4106d20 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1335680809 0 0) 0x7fd8ac0035b0 con 0x7fd8c410aa60 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8c41a53e0 con 0x7fd8c410aa60 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8bc001bf0 con 0x7fd8c410aa60 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2332216607 0 0) 0x7fd8c4198a10 con 0x7fd8c4106d20 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8ac0035b0 con 0x7fd8c4106d20 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1008730918 0 0) 0x7fd8c41a4200 con 0x7fd8c41028f0 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8c4198a10 con 0x7fd8c41028f0 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8b4001430 con 0x7fd8c4106d20 2024-04-14T15:09:59.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8b8001720 con 0x7fd8c41028f0 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 366249849 0 0) 0x7fd8c41a53e0 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 >> v1:172.21.15.45:6790/0 conn(0x7fd8c4106d20 legacy=0x7fd8c419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 >> v1:172.21.15.45:6789/0 conn(0x7fd8c41028f0 legacy=0x7fd8c4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c41a6600 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c41a4430 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd8c41a4970 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd8bc003eb0 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.727+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8bc006030 con 0x7fd8c410aa60 2024-04-14T15:09:59.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.728+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a4005150 con 0x7fd8c410aa60 2024-04-14T15:09:59.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.728+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7fd8bc003a20 con 0x7fd8c410aa60 2024-04-14T15:09:59.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.729+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(59..59 src has 1..59) ==== 4081+0+0 (unknown 1303600178 0 0) 0x7fd8bc089d50 con 0x7fd8c410aa60 2024-04-14T15:09:59.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.734+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fd8bc057750 con 0x7fd8c410aa60 2024-04-14T15:09:59.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:09:59 smithi045 systemd[1]: Starting Ceph rgw.foo.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:09:59.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:09:59.932+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) -- 0x7fd8a4005440 con 0x7fd8c410aa60 2024-04-14T15:10:00.035 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:09:59 smithi045 podman[64122]: 2024-04-14 15:09:59.660346533 +0000 UTC m=+0.017694029 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:10:00.036 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:09:59 smithi045 podman[64122]: 2024-04-14 15:09:59.791255964 +0000 UTC m=+0.148603503 container create f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=) 2024-04-14T15:10:00.036 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:10:00 smithi045 podman[64122]: 2024-04-14 15:10:00.03571455 +0000 UTC m=+0.393062044 container init f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-14T15:10:00.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.271+0000 7fd8b37fe700 1 -- 172.21.15.71:0/626713869 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v60) ==== 160+0+0 (unknown 701681216 0 0) 0x7fd8bc057050 con 0x7fd8c410aa60 2024-04-14T15:10:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 >> v1:172.21.15.45:6800/2191709230 conn(0x7fd8ac06f950 legacy=0x7fd8ac071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 >> v1:172.21.15.71:6789/0 conn(0x7fd8c410aa60 legacy=0x7fd8c41a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 shutdown_connections 2024-04-14T15:10:00.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 >> 172.21.15.71:0/626713869 conn(0x7fd8c40fdeb0 msgr2=0x7fd8c4109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 shutdown_connections 2024-04-14T15:10:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:00.275+0000 7fd8c8c67700 1 -- 172.21.15.71:0/626713869 wait complete. 2024-04-14T15:10:00.276 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'datapool' created 2024-04-14T15:10:00.497 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:10:00 smithi045 podman[64122]: 2024-04-14 15:10:00.038735158 +0000 UTC m=+0.396082651 container start f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:10:00.497 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:10:00 smithi045 bash[64122]: f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 2024-04-14T15:10:00.497 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:10:00 smithi045 systemd[1]: Started Ceph rgw.foo.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/626713869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: overall HEALTH_OK 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/626713869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: overall HEALTH_OK 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:00 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/626713869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='client.24361 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: overall HEALTH_OK 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:00 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:00.866 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- rbd pool init datapool 2024-04-14T15:10:01.102 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: osdmap e60: 8 total, 8 up, 8 in 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3203953394' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: from='client.34352 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: osdmap e60: 8 total, 8 up, 8 in 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3203953394' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: from='client.34352 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:01 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:01.560 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: Saving service rgw.foo.a spec with placement smithi045=foo.a;count:1 2024-04-14T15:10:01.560 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: from='client.24361 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-04-14T15:10:01.560 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: osdmap e60: 8 total, 8 up, 8 in 2024-04-14T15:10:01.561 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3203953394' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.561 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: from='client.34352 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-04-14T15:10:01.561 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:01 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:02.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: pgmap v181: 36 pgs: 4 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:10:02.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:10:02.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: from='client.34352 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-04-14T15:10:02.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: osdmap e61: 8 total, 8 up, 8 in 2024-04-14T15:10:02.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:02.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:02 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: pgmap v181: 36 pgs: 4 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: from='client.34352 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: osdmap e61: 8 total, 8 up, 8 in 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: pgmap v181: 36 pgs: 4 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: from='client.34352 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: osdmap e61: 8 total, 8 up, 8 in 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:02 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: Checking dashboard <-> RGW credentials 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: osdmap e62: 8 total, 8 up, 8 in 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1852317703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:03 smithi071 ceph-mon[33150]: from='client.24415 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: Checking dashboard <-> RGW credentials 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: osdmap e62: 8 total, 8 up, 8 in 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1852317703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[29624]: from='client.24415 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: Checking dashboard <-> RGW credentials 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: osdmap e62: 8 total, 8 up, 8 in 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1852317703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:03 smithi045 ceph-mon[37795]: from='client.24415 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: pgmap v184: 68 pgs: 4 creating+peering, 1 active+clean, 63 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: from='client.24415 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: osdmap e63: 8 total, 8 up, 8 in 2024-04-14T15:10:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:04 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: pgmap v184: 68 pgs: 4 creating+peering, 1 active+clean, 63 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: from='client.24415 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: osdmap e63: 8 total, 8 up, 8 in 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: pgmap v184: 68 pgs: 4 creating+peering, 1 active+clean, 63 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: from='client.24415 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: osdmap e63: 8 total, 8 up, 8 in 2024-04-14T15:10:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:04 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:05 smithi071 ceph-mon[33150]: osdmap e64: 8 total, 8 up, 8 in 2024-04-14T15:10:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:05 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[37795]: osdmap e64: 8 total, 8 up, 8 in 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[29624]: osdmap e64: 8 total, 8 up, 8 in 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-04-14T15:10:05.889 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.71 --placement '1;smithi071=iscsi.a' 2024-04-14T15:10:06.132 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:06.517 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:06 smithi071 ceph-mon[33150]: pgmap v187: 100 pgs: 2 creating+activating, 11 creating+peering, 23 active+clean, 64 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-14T15:10:06.517 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:06 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.517 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.517 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:06 smithi071 ceph-mon[33150]: osdmap e65: 8 total, 8 up, 8 in 2024-04-14T15:10:06.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[29624]: pgmap v187: 100 pgs: 2 creating+activating, 11 creating+peering, 23 active+clean, 64 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-14T15:10:06.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[29624]: osdmap e65: 8 total, 8 up, 8 in 2024-04-14T15:10:06.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[37795]: pgmap v187: 100 pgs: 2 creating+activating, 11 creating+peering, 23 active+clean, 64 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-04-14T15:10:06.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-04-14T15:10:06.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:06 smithi045 ceph-mon[37795]: osdmap e65: 8 total, 8 up, 8 in 2024-04-14T15:10:07.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:07 smithi071 ceph-mon[33150]: osdmap e66: 8 total, 8 up, 8 in 2024-04-14T15:10:07.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:07 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[37795]: osdmap e66: 8 total, 8 up, 8 in 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[29624]: osdmap e66: 8 total, 8 up, 8 in 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:07 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-04-14T15:10:07.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.875+0000 7f0436bd0700 1 -- 172.21.15.71:0/2295515855 >> v1:172.21.15.71:6789/0 conn(0x7f0430106d10 legacy=0x7f0430109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:07.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.875+0000 7f0436bd0700 1 -- 172.21.15.71:0/2295515855 shutdown_connections 2024-04-14T15:10:07.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.875+0000 7f0436bd0700 1 -- 172.21.15.71:0/2295515855 >> 172.21.15.71:0/2295515855 conn(0x7f04300fde80 msgr2=0x7f04301002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.875+0000 7f0436bd0700 1 -- 172.21.15.71:0/2295515855 shutdown_connections 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 -- 172.21.15.71:0/2295515855 wait complete. 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 Processor -- start 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 -- start start 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0430198bb0 con 0x7f04301028e0 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f04301a43a0 con 0x7f043010aa50 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f0436bd0700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f04301a5580 con 0x7f0430106d10 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f042ffff700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f0430106d10 0x7f043019e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.71:50452/0 (socket says 172.21.15.71:50452) 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f042ffff700 1 -- 172.21.15.71:0/4196754733 learned_addr learned my addr 172.21.15.71:0/4196754733 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:07.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.876+0000 7f043516d700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f043010aa50 0x7f04301a1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:38862/0 (socket says 172.21.15.71:38862) 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968646986 0 0) 0x7f04301a43a0 con 0x7f043010aa50 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04200035b0 con 0x7f043010aa50 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224186404 0 0) 0x7f04301a5580 con 0x7f0430106d10 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04301a43a0 con 0x7f0430106d10 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444669792 0 0) 0x7f0430198bb0 con 0x7f04301028e0 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04301a5580 con 0x7f04301028e0 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3627858031 0 0) 0x7f04200035b0 con 0x7f043010aa50 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0430198bb0 con 0x7f043010aa50 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0428001bb0 con 0x7f043010aa50 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3553611058 0 0) 0x7f04301a43a0 con 0x7f0430106d10 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04200035b0 con 0x7f0430106d10 2024-04-14T15:10:07.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f041c001430 con 0x7f0430106d10 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2346082515 0 0) 0x7f04301a5580 con 0x7f04301028e0 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04301a43a0 con 0x7f04301028e0 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f04240017f0 con 0x7f04301028e0 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2499208750 0 0) 0x7f0430198bb0 con 0x7f043010aa50 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 >> v1:172.21.15.45:6790/0 conn(0x7f0430106d10 legacy=0x7f043019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 >> v1:172.21.15.45:6789/0 conn(0x7f04301028e0 legacy=0x7f0430198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04301a67a0 con 0x7f043010aa50 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f04301a45d0 con 0x7f043010aa50 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0428003e90 con 0x7f043010aa50 2024-04-14T15:10:07.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0428006020 con 0x7f043010aa50 2024-04-14T15:10:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.877+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f04301a4b10 con 0x7f043010aa50 2024-04-14T15:10:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.878+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430110460 con 0x7f043010aa50 2024-04-14T15:10:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.879+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7f0428003a00 con 0x7f043010aa50 2024-04-14T15:10:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.879+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(67..67 src has 1..67) ==== 5926+0+0 (unknown 262799028 0 0) 0x7f042808a210 con 0x7f043010aa50 2024-04-14T15:10:07.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:07.884+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0428057540 con 0x7f043010aa50 2024-04-14T15:10:08.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.091+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.71", "placement": "1;smithi071=iscsi.a", "target": ["mon-mgr", ""]}) -- 0x7f04301a4e00 con 0x7f042006fba0 2024-04-14T15:10:08.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.099+0000 7f042dffb700 1 -- 172.21.15.71:0/4196754733 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (unknown 0 0 803663096) 0x7f04301a4e00 con 0x7f042006fba0 2024-04-14T15:10:08.099 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled iscsi.datapool update... 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 >> v1:172.21.15.45:6800/2191709230 conn(0x7f042006fba0 legacy=0x7f0420072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 >> v1:172.21.15.71:6789/0 conn(0x7f043010aa50 legacy=0x7f04301a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 shutdown_connections 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 >> 172.21.15.71:0/4196754733 conn(0x7f04300fde80 msgr2=0x7f0430109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 shutdown_connections 2024-04-14T15:10:08.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:08.101+0000 7f0436bd0700 1 -- 172.21.15.71:0/4196754733 wait complete. 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: pgmap v190: 132 pgs: 2 creating+activating, 17 creating+peering, 63 active+clean, 50 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: osdmap e67: 8 total, 8 up, 8 in 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:08 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:08.734 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-04-14T15:10:08.734 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:10:08.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-04-14T15:10:08.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a[64137]: 2024-04-14T15:10:08.365+0000 7f84571bbac0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-14T15:10:08.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: pgmap v190: 132 pgs: 2 creating+activating, 17 creating+peering, 63 active+clean, 50 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:10:08.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: osdmap e67: 8 total, 8 up, 8 in 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: pgmap v190: 132 pgs: 2 creating+activating, 17 creating+peering, 63 active+clean, 50 unknown; 578 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: osdmap e67: 8 total, 8 up, 8 in 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1686617690' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-04-14T15:10:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:08 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:08.766 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:10:08.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-04-14T15:10:08.794 DEBUG:teuthology.orchestra.run.smithi071:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@iscsi.iscsi.a.service 2024-04-14T15:10:08.837 INFO:tasks.cephadm:Adding prometheus.a on smithi071 2024-04-14T15:10:08.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply prometheus '1;smithi071=a' 2024-04-14T15:10:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='client.24421 v1:172.21.15.71:0/4196754733' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.71", "placement": "1;smithi071=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: Saving service iscsi.datapool spec with placement smithi071=iscsi.a;count:1 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: osdmap e68: 8 total, 8 up, 8 in 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-04-14T15:10:09.543 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:09 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='client.24421 v1:172.21.15.71:0/4196754733' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.71", "placement": "1;smithi071=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: Saving service iscsi.datapool spec with placement smithi071=iscsi.a;count:1 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: osdmap e68: 8 total, 8 up, 8 in 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='client.24421 v1:172.21.15.71:0/4196754733' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.71", "placement": "1;smithi071=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: Saving service iscsi.datapool spec with placement smithi071=iscsi.a;count:1 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/417276953' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='client.24409 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: osdmap e68: 8 total, 8 up, 8 in 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-04-14T15:10:09.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:09 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:10 smithi071 ceph-mon[33150]: pgmap v193: 132 pgs: 10 creating+peering, 106 active+clean, 16 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 4.2 KiB/s wr, 53 op/s 2024-04-14T15:10:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:10 smithi071 ceph-mon[33150]: Deploying daemon iscsi.iscsi.a on smithi071 2024-04-14T15:10:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:10 smithi071 ceph-mon[33150]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-14T15:10:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:10 smithi071 ceph-mon[33150]: Cluster is now healthy 2024-04-14T15:10:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[29624]: pgmap v193: 132 pgs: 10 creating+peering, 106 active+clean, 16 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 4.2 KiB/s wr, 53 op/s 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[29624]: Deploying daemon iscsi.iscsi.a on smithi071 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[29624]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[29624]: Cluster is now healthy 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[37795]: pgmap v193: 132 pgs: 10 creating+peering, 106 active+clean, 16 unknown; 579 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 4.2 KiB/s wr, 53 op/s 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[37795]: Deploying daemon iscsi.iscsi.a on smithi071 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[37795]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-14T15:10:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:10 smithi045 ceph-mon[37795]: Cluster is now healthy 2024-04-14T15:10:11.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.735+0000 7fb159de5700 1 -- 172.21.15.71:0/1922120151 >> v1:172.21.15.71:6789/0 conn(0x7fb154105c60 legacy=0x7fb1541080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:11.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.735+0000 7fb159de5700 1 -- 172.21.15.71:0/1922120151 shutdown_connections 2024-04-14T15:10:11.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.735+0000 7fb159de5700 1 -- 172.21.15.71:0/1922120151 >> 172.21.15.71:0/1922120151 conn(0x7fb1540fcdd0 msgr2=0x7fb1540ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:11.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.735+0000 7fb159de5700 1 -- 172.21.15.71:0/1922120151 shutdown_connections 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.735+0000 7fb159de5700 1 -- 172.21.15.71:0/1922120151 wait complete. 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb159de5700 1 Processor -- start 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb159de5700 1 -- start start 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb159de5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb15406bd10 con 0x7fb154101830 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb159de5700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1541a4180 con 0x7fb1541099a0 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb159de5700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fb1541a53c0 con 0x7fb154105c60 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb153fff700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fb1541099a0 0x7fb1541a1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:38882/0 (socket says 172.21.15.71:38882) 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.736+0000 7fb153fff700 1 -- 172.21.15.71:0/1062701993 learned_addr learned my addr 172.21.15.71:0/1062701993 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154298211 0 0) 0x7fb1541a4180 con 0x7fb1541099a0 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb13c0035b0 con 0x7fb1541099a0 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884567561 0 0) 0x7fb1541a53c0 con 0x7fb154105c60 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1541a4180 con 0x7fb154105c60 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810982574 0 0) 0x7fb15406bd10 con 0x7fb154101830 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1541a53c0 con 0x7fb154101830 2024-04-14T15:10:11.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4157132195 0 0) 0x7fb13c0035b0 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb15406bd10 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fb14c001c10 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2632407856 0 0) 0x7fb1541a53c0 con 0x7fb154101830 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb13c0035b0 con 0x7fb154101830 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4017799977 0 0) 0x7fb1541a4180 con 0x7fb154105c60 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb1541a53c0 con 0x7fb154105c60 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fb1480017f0 con 0x7fb154101830 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fb144001430 con 0x7fb154105c60 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2929689719 0 0) 0x7fb15406bd10 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 >> v1:172.21.15.45:6790/0 conn(0x7fb154105c60 legacy=0x7fb15419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 >> v1:172.21.15.45:6789/0 conn(0x7fb154101830 legacy=0x7fb15419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1541a6600 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb1541a55f0 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb1541a5b30 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fb14c003eb0 con 0x7fb1541099a0 2024-04-14T15:10:11.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.737+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fb14c005f30 con 0x7fb1541099a0 2024-04-14T15:10:11.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.738+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1540693d0 con 0x7fb1541099a0 2024-04-14T15:10:11.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.739+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 15) ==== 92623+0+0 (unknown 736615685 0 0) 0x7fb14c01cb70 con 0x7fb1541099a0 2024-04-14T15:10:11.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.740+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(68..68 src has 1..68) ==== 5942+0+0 (unknown 1850546898 0 0) 0x7fb14c08a3e0 con 0x7fb1541099a0 2024-04-14T15:10:11.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.745+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fb14c057700 con 0x7fb1541099a0 2024-04-14T15:10:11.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.965+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}) -- 0x7fb1541a5e90 con 0x7fb13c06f930 2024-04-14T15:10:11.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.973+0000 7fb150ff9700 1 -- 172.21.15.71:0/1062701993 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+31 (unknown 0 0 1342662408) 0x7fb1541a5e90 con 0x7fb13c06f930 2024-04-14T15:10:11.973 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled prometheus update... 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 >> v1:172.21.15.45:6800/2191709230 conn(0x7fb13c06f930 legacy=0x7fb13c071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 >> v1:172.21.15.71:6789/0 conn(0x7fb1541099a0 legacy=0x7fb1541a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 shutdown_connections 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 >> 172.21.15.71:0/1062701993 conn(0x7fb1540fcdd0 msgr2=0x7fb1541080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 shutdown_connections 2024-04-14T15:10:11.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:11.975+0000 7fb159de5700 1 -- 172.21.15.71:0/1062701993 wait complete. 2024-04-14T15:10:12.553 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:12 smithi071 ceph-mon[33150]: pgmap v194: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.9 KiB/s wr, 148 op/s 2024-04-14T15:10:12.553 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:12 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:12 smithi045 ceph-mon[29624]: pgmap v194: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.9 KiB/s wr, 148 op/s 2024-04-14T15:10:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:12 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:12 smithi045 ceph-mon[37795]: pgmap v194: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 5.9 KiB/s wr, 148 op/s 2024-04-14T15:10:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:12 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:12.812 DEBUG:teuthology.orchestra.run.smithi071:prometheus.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@prometheus.a.service 2024-04-14T15:10:12.814 INFO:tasks.cephadm:Adding node-exporter.a on smithi045 2024-04-14T15:10:12.815 INFO:tasks.cephadm:Adding node-exporter.b on smithi071 2024-04-14T15:10:12.815 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply node-exporter '2;smithi045=a;smithi071=b' 2024-04-14T15:10:13.436 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='client.24442 v1:172.21.15.71:0/1062701993' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: Saving service prometheus spec with placement smithi071=a;count:1 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: Checking pool "datapool" exists for service iscsi.datapool 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:13.725 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1591171651' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-14T15:10:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='client.24442 v1:172.21.15.71:0/1062701993' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: Saving service prometheus spec with placement smithi071=a;count:1 2024-04-14T15:10:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: Checking pool "datapool" exists for service iscsi.datapool 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1591171651' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='client.24442 v1:172.21.15.71:0/1062701993' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: Saving service prometheus spec with placement smithi071=a;count:1 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: Checking pool "datapool" exists for service iscsi.datapool 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1591171651' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-14T15:10:14.800 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:14 smithi071 ceph-mon[33150]: pgmap v195: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 5.0 KiB/s wr, 125 op/s 2024-04-14T15:10:14.800 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:14 smithi071 ceph-mon[33150]: mgrmap e16: y(active, since 4m), standbys: x 2024-04-14T15:10:14.800 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:14 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:14.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[37795]: pgmap v195: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 5.0 KiB/s wr, 125 op/s 2024-04-14T15:10:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[37795]: mgrmap e16: y(active, since 4m), standbys: x 2024-04-14T15:10:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[29624]: pgmap v195: 132 pgs: 132 active+clean; 581 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s rd, 5.0 KiB/s wr, 125 op/s 2024-04-14T15:10:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[29624]: mgrmap e16: y(active, since 4m), standbys: x 2024-04-14T15:10:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:14 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.575+0000 7f9a96647700 1 -- 172.21.15.71:0/39031237 >> v1:172.21.15.71:6789/0 conn(0x7f9a901053b0 legacy=0x7f9a901077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.575+0000 7f9a96647700 1 -- 172.21.15.71:0/39031237 shutdown_connections 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.575+0000 7f9a96647700 1 -- 172.21.15.71:0/39031237 >> 172.21.15.71:0/39031237 conn(0x7f9a900fc520 msgr2=0x7f9a900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.575+0000 7f9a96647700 1 -- 172.21.15.71:0/39031237 shutdown_connections 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.575+0000 7f9a96647700 1 -- 172.21.15.71:0/39031237 wait complete. 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a96647700 1 Processor -- start 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a96647700 1 -- start start 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a96647700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9a901115a0 con 0x7f9a901053b0 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a96647700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9a901a4200 con 0x7f9a90100f10 2024-04-14T15:10:15.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a96647700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9a901a53e0 con 0x7f9a90109040 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a8ffff700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f9a90100f10 0x7f9a9019ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48410/0 (socket says 172.21.15.71:48410) 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.576+0000 7f9a8ffff700 1 -- 172.21.15.71:0/1783659532 learned_addr learned my addr 172.21.15.71:0/1783659532 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484291300 0 0) 0x7f9a901a4200 con 0x7f9a90100f10 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a780035b0 con 0x7f9a90100f10 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720459479 0 0) 0x7f9a901a53e0 con 0x7f9a90109040 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a901a4200 con 0x7f9a90109040 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302507132 0 0) 0x7f9a901115a0 con 0x7f9a901053b0 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a901a53e0 con 0x7f9a901053b0 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2013183068 0 0) 0x7f9a780035b0 con 0x7f9a90100f10 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a901115a0 con 0x7f9a90100f10 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9a800018b0 con 0x7f9a90100f10 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2936387040 0 0) 0x7f9a901a4200 con 0x7f9a90109040 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a780035b0 con 0x7f9a90109040 2024-04-14T15:10:15.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9a88001af0 con 0x7f9a90109040 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1526348938 0 0) 0x7f9a901a53e0 con 0x7f9a901053b0 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a901a4200 con 0x7f9a901053b0 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9a84001430 con 0x7f9a901053b0 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2782250733 0 0) 0x7f9a901115a0 con 0x7f9a90100f10 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 >> v1:172.21.15.45:6790/0 conn(0x7f9a90109040 legacy=0x7f9a901a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 >> v1:172.21.15.45:6789/0 conn(0x7f9a901053b0 legacy=0x7f9a9019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a901a6620 con 0x7f9a90100f10 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9a901a4430 con 0x7f9a90100f10 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9a901a4970 con 0x7f9a90100f10 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9a80003b30 con 0x7f9a90100f10 2024-04-14T15:10:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.577+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9a80005c90 con 0x7f9a90100f10 2024-04-14T15:10:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.578+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7c005150 con 0x7f9a90100f10 2024-04-14T15:10:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.579+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 16) ==== 92674+0+0 (unknown 428266710 0 0) 0x7f9a8001c680 con 0x7f9a90100f10 2024-04-14T15:10:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.579+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f9a8008a270 con 0x7f9a90100f10 2024-04-14T15:10:15.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.584+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9a8005fa00 con 0x7f9a90100f10 2024-04-14T15:10:15.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.799+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi045=a;smithi071=b", "target": ["mon-mgr", ""]}) -- 0x7f9a7c000bc0 con 0x7f9a7806fa40 2024-04-14T15:10:15.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.806+0000 7f9a8d7fa700 1 -- 172.21.15.71:0/1783659532 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+34 (unknown 0 0 240551134) 0x7f9a7c000bc0 con 0x7f9a7806fa40 2024-04-14T15:10:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled node-exporter update... 2024-04-14T15:10:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9a7806fa40 legacy=0x7f9a78071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 >> v1:172.21.15.71:6789/0 conn(0x7f9a90100f10 legacy=0x7f9a9019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:15.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 shutdown_connections 2024-04-14T15:10:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 >> 172.21.15.71:0/1783659532 conn(0x7f9a900fc520 msgr2=0x7f9a9010c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 shutdown_connections 2024-04-14T15:10:15.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:15.808+0000 7f9a96647700 1 -- 172.21.15.71:0/1783659532 wait complete. 2024-04-14T15:10:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:15 smithi071 ceph-mon[33150]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-04-14T15:10:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:15 smithi071 ceph-mon[33150]: osdmap e69: 8 total, 8 up, 8 in 2024-04-14T15:10:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:15 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[37795]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-04-14T15:10:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[37795]: osdmap e69: 8 total, 8 up, 8 in 2024-04-14T15:10:16.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[29624]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-04-14T15:10:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[29624]: osdmap e69: 8 total, 8 up, 8 in 2024-04-14T15:10:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:15 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:16.876 DEBUG:teuthology.orchestra.run.smithi045:node-exporter.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.a.service 2024-04-14T15:10:16.879 DEBUG:teuthology.orchestra.run.smithi071:node-exporter.b> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.b.service 2024-04-14T15:10:16.881 INFO:tasks.cephadm:Adding alertmanager.a on smithi045 2024-04-14T15:10:16.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply alertmanager '1;smithi045=a' 2024-04-14T15:10:17.171 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='client.24478 v1:172.21.15.71:0/1783659532' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi045=a;smithi071=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: Saving service node-exporter spec with placement smithi045=a;smithi071=b;count:2 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:17 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='client.24478 v1:172.21.15.71:0/1783659532' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi045=a;smithi071=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: Saving service node-exporter spec with placement smithi045=a;smithi071=b;count:2 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='client.24478 v1:172.21.15.71:0/1783659532' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi045=a;smithi071=b", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: Saving service node-exporter spec with placement smithi045=a;smithi071=b;count:2 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:10:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:10:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:17 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:18 smithi071 ceph-mon[33150]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-04-14T15:10:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:18 smithi071 ceph-mon[33150]: Deploying daemon prometheus.a on smithi071 2024-04-14T15:10:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:18 smithi045 ceph-mon[29624]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-04-14T15:10:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:18 smithi045 ceph-mon[29624]: Deploying daemon prometheus.a on smithi071 2024-04-14T15:10:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:18 smithi045 ceph-mon[37795]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.7 KiB/s wr, 138 op/s 2024-04-14T15:10:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:18 smithi045 ceph-mon[37795]: Deploying daemon prometheus.a on smithi071 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.758+0000 7f0af0b6d700 1 -- 172.21.15.71:0/2699631649 >> v1:172.21.15.71:6789/0 conn(0x7f0aec1017e0 legacy=0x7f0aec101c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.758+0000 7f0af0b6d700 1 -- 172.21.15.71:0/2699631649 shutdown_connections 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.758+0000 7f0af0b6d700 1 -- 172.21.15.71:0/2699631649 >> 172.21.15.71:0/2699631649 conn(0x7f0aec0fcd80 msgr2=0x7f0aec0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- 172.21.15.71:0/2699631649 shutdown_connections 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- 172.21.15.71:0/2699631649 wait complete. 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 Processor -- start 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- start start 2024-04-14T15:10:18.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0aec198a60 con 0x7f0aec109950 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0aec1a4250 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.759+0000 7f0af0b6d700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f0aec1a5430 con 0x7f0aec105c10 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0aea59c700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f0aec1017e0 0x7f0aec197f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48436/0 (socket says 172.21.15.71:48436) 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0aea59c700 1 -- 172.21.15.71:0/3615799588 learned_addr learned my addr 172.21.15.71:0/3615799588 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1211879942 0 0) 0x7f0aec1a4250 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad40035b0 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794018540 0 0) 0x7f0aec1a5430 con 0x7f0aec105c10 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0aec1a4250 con 0x7f0aec105c10 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158383305 0 0) 0x7f0aec198a60 con 0x7f0aec109950 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0aec1a5430 con 0x7f0aec109950 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2032491391 0 0) 0x7f0ad40035b0 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0aec198a60 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0adc001430 con 0x7f0aec1017e0 2024-04-14T15:10:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3795789150 0 0) 0x7f0aec1a4250 con 0x7f0aec105c10 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ad40035b0 con 0x7f0aec105c10 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0ae0001830 con 0x7f0aec105c10 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2239043109 0 0) 0x7f0aec1a5430 con 0x7f0aec109950 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0aec1a4250 con 0x7f0aec109950 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 842338493 0 0) 0x7f0aec198a60 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 >> v1:172.21.15.45:6790/0 conn(0x7f0aec105c10 legacy=0x7f0aec19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 >> v1:172.21.15.45:6789/0 conn(0x7f0aec109950 legacy=0x7f0aec1a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec1a6650 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec1a5660 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f0aec1a5ba0 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0adc001690 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.760+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0adc004eb0 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.761+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc005150 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.762+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 16) ==== 92674+0+0 (unknown 428266710 0 0) 0x7f0adc003ab0 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.762+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f0adc08a2f0 con 0x7f0aec1017e0 2024-04-14T15:10:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.767+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0adc057510 con 0x7f0aec1017e0 2024-04-14T15:10:18.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.980+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi045=a", "target": ["mon-mgr", ""]}) -- 0x7f0acc000bc0 con 0x7f0ad406f990 2024-04-14T15:10:18.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.987+0000 7f0adb7fe700 1 -- 172.21.15.71:0/3615799588 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+33 (unknown 0 0 1850065467) 0x7f0acc000bc0 con 0x7f0ad406f990 2024-04-14T15:10:18.987 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled alertmanager update... 2024-04-14T15:10:18.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 >> v1:172.21.15.45:6800/2191709230 conn(0x7f0ad406f990 legacy=0x7f0ad4071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:18.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 >> v1:172.21.15.71:6789/0 conn(0x7f0aec1017e0 legacy=0x7f0aec197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:18.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 shutdown_connections 2024-04-14T15:10:18.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 >> 172.21.15.71:0/3615799588 conn(0x7f0aec0fcd80 msgr2=0x7f0aec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:18.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 shutdown_connections 2024-04-14T15:10:18.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:18.989+0000 7f0af0b6d700 1 -- 172.21.15.71:0/3615799588 wait complete. 2024-04-14T15:10:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:19 smithi071 ceph-mon[33150]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.5 KiB/s wr, 93 op/s 2024-04-14T15:10:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:19 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:19 smithi071 ceph-mon[33150]: from='client.24484 v1:172.21.15.71:0/3615799588' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi045=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:19 smithi071 ceph-mon[33150]: Saving service alertmanager spec with placement smithi045=a;count:1 2024-04-14T15:10:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:19 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[37795]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.5 KiB/s wr, 93 op/s 2024-04-14T15:10:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[37795]: from='client.24484 v1:172.21.15.71:0/3615799588' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi045=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[37795]: Saving service alertmanager spec with placement smithi045=a;count:1 2024-04-14T15:10:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:19.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[29624]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.5 KiB/s wr, 93 op/s 2024-04-14T15:10:19.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:19.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[29624]: from='client.24484 v1:172.21.15.71:0/3615799588' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi045=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:19.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[29624]: Saving service alertmanager spec with placement smithi045=a;count:1 2024-04-14T15:10:19.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:19 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:20.386 DEBUG:teuthology.orchestra.run.smithi045:alertmanager.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@alertmanager.a.service 2024-04-14T15:10:20.389 INFO:tasks.cephadm:Adding grafana.a on smithi071 2024-04-14T15:10:20.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph orch apply grafana '1;smithi071=a' 2024-04-14T15:10:20.667 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:21.855 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:21 smithi071 ceph-mon[33150]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:21.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:21 smithi045 ceph-mon[29624]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:21.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:21 smithi045 ceph-mon[37795]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:22.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.914+0000 7fe453587700 1 -- 172.21.15.71:0/2603237007 >> v1:172.21.15.71:6789/0 conn(0x7fe44c10aa80 legacy=0x7fe44c10cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.914+0000 7fe453587700 1 -- 172.21.15.71:0/2603237007 shutdown_connections 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.914+0000 7fe453587700 1 -- 172.21.15.71:0/2603237007 >> 172.21.15.71:0/2603237007 conn(0x7fe44c0fded0 msgr2=0x7fe44c1002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.914+0000 7fe453587700 1 -- 172.21.15.71:0/2603237007 shutdown_connections 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.914+0000 7fe453587700 1 -- 172.21.15.71:0/2603237007 wait complete. 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe453587700 1 Processor -- start 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe453587700 1 -- start start 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe453587700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe44c198c20 con 0x7fe44c106d40 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe453587700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe44c1a4410 con 0x7fe44c10aa80 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe453587700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fe44c1a55f0 con 0x7fe44c102910 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe451b24700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7fe44c10aa80 0x7fe44c1a1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:48464/0 (socket says 172.21.15.71:48464) 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe451b24700 1 -- 172.21.15.71:0/4108261471 learned_addr learned my addr 172.21.15.71:0/4108261471 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985549114 0 0) 0x7fe44c1a4410 con 0x7fe44c10aa80 2024-04-14T15:10:22.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4380035b0 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203142915 0 0) 0x7fe44c198c20 con 0x7fe44c106d40 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe44c1a4410 con 0x7fe44c106d40 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3286440786 0 0) 0x7fe44c1a55f0 con 0x7fe44c102910 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.915+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe44c198c20 con 0x7fe44c102910 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4252533339 0 0) 0x7fe4380035b0 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe44c1a55f0 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe4440016f0 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3216659068 0 0) 0x7fe44c1a4410 con 0x7fe44c106d40 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe4380035b0 con 0x7fe44c106d40 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe43c001720 con 0x7fe44c106d40 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3103655809 0 0) 0x7fe44c1a55f0 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 >> v1:172.21.15.45:6790/0 conn(0x7fe44c102910 legacy=0x7fe44c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 >> v1:172.21.15.45:6789/0 conn(0x7fe44c106d40 legacy=0x7fe44c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c1a6810 con 0x7fe44c10aa80 2024-04-14T15:10:22.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c1a3460 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.916+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe44c1a39a0 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.917+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fe444003c10 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.917+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fe444004f80 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.917+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe430005120 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.918+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 16) ==== 92674+0+0 (unknown 428266710 0 0) 0x7fe444004f80 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.919+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7fe44408a440 con 0x7fe44c10aa80 2024-04-14T15:10:22.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:22.928+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fe444057660 con 0x7fe44c10aa80 2024-04-14T15:10:23.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.183+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 --> v1:172.21.15.45:6800/2191709230 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}) -- 0x7fe430000b90 con 0x7fe43806f930 2024-04-14T15:10:23.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.190+0000 7fe4427fc700 1 -- 172.21.15.71:0/4108261471 <== mgr.14152 v1:172.21.15.45:6800/2191709230 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (unknown 0 0 664801700) 0x7fe430000b90 con 0x7fe43806f930 2024-04-14T15:10:23.191 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled grafana update... 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.192+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 >> v1:172.21.15.45:6800/2191709230 conn(0x7fe43806f930 legacy=0x7fe438071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.192+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 >> v1:172.21.15.71:6789/0 conn(0x7fe44c10aa80 legacy=0x7fe44c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.193+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 shutdown_connections 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.193+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 >> 172.21.15.71:0/4108261471 conn(0x7fe44c0fded0 msgr2=0x7fe44c10d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.193+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 shutdown_connections 2024-04-14T15:10:23.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:23.193+0000 7fe453587700 1 -- 172.21.15.71:0/4108261471 wait complete. 2024-04-14T15:10:24.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:24 smithi071 ceph-mon[33150]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:24.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:24.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:24 smithi071 ceph-mon[33150]: from='client.24490 v1:172.21.15.71:0/4108261471' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:24.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:24 smithi071 ceph-mon[33150]: Saving service grafana spec with placement smithi071=a;count:1 2024-04-14T15:10:24.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:24 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[29624]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[29624]: from='client.24490 v1:172.21.15.71:0/4108261471' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[29624]: Saving service grafana spec with placement smithi071=a;count:1 2024-04-14T15:10:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[37795]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 818 B/s wr, 36 op/s 2024-04-14T15:10:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[37795]: from='client.24490 v1:172.21.15.71:0/4108261471' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-14T15:10:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[37795]: Saving service grafana spec with placement smithi071=a;count:1 2024-04-14T15:10:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:24 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:24.498 DEBUG:teuthology.orchestra.run.smithi071:grafana.a> sudo journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@grafana.a.service 2024-04-14T15:10:24.500 INFO:tasks.cephadm:Setting up client nodes... 2024-04-14T15:10:24.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-14T15:10:24.756 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:26.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:26 smithi045 ceph-mon[29624]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:10:26.442 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:26 smithi045 ceph-mon[37795]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.559+0000 7f00c7284700 1 -- 172.21.15.45:0/2900544704 >> v1:172.21.15.45:6790/0 conn(0x7f00c0106d10 legacy=0x7f00c0109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.559+0000 7f00c7284700 1 -- 172.21.15.45:0/2900544704 shutdown_connections 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.559+0000 7f00c7284700 1 -- 172.21.15.45:0/2900544704 >> 172.21.15.45:0/2900544704 conn(0x7f00c00fde80 msgr2=0x7f00c01002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.559+0000 7f00c7284700 1 -- 172.21.15.45:0/2900544704 shutdown_connections 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.559+0000 7f00c7284700 1 -- 172.21.15.45:0/2900544704 wait complete. 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c7284700 1 Processor -- start 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c7284700 1 -- start start 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c7284700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f00c0198810 con 0x7f00c01028e0 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c7284700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f00c01a4000 con 0x7f00c0106d10 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c7284700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f00c01a51e0 con 0x7f00c010aa50 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c5020700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f00c01028e0 0x7f00c0197cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58880/0 (socket says 172.21.15.45:58880) 2024-04-14T15:10:26.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00c5020700 1 -- 172.21.15.45:0/3258445934 learned_addr learned my addr 172.21.15.45:0/3258445934 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607461130 0 0) 0x7f00c01a51e0 con 0x7f00c010aa50 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.560+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00ac0035b0 con 0x7f00c010aa50 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194501032 0 0) 0x7f00c0198810 con 0x7f00c01028e0 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00c01a51e0 con 0x7f00c01028e0 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627667543 0 0) 0x7f00c01a4000 con 0x7f00c0106d10 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00c0198810 con 0x7f00c0106d10 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 691749818 0 0) 0x7f00ac0035b0 con 0x7f00c010aa50 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00c01a4000 con 0x7f00c010aa50 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f00b8001c30 con 0x7f00c010aa50 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1148102507 0 0) 0x7f00c0198810 con 0x7f00c0106d10 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00ac0035b0 con 0x7f00c0106d10 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f00b0004430 con 0x7f00c0106d10 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 916860073 0 0) 0x7f00c01a51e0 con 0x7f00c01028e0 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00c0198810 con 0x7f00c01028e0 2024-04-14T15:10:26.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f00bc001870 con 0x7f00c01028e0 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3210588071 0 0) 0x7f00ac0035b0 con 0x7f00c0106d10 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 >> v1:172.21.15.45:6790/0 conn(0x7f00c010aa50 legacy=0x7f00c01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 >> v1:172.21.15.45:6789/0 conn(0x7f00c01028e0 legacy=0x7f00c0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c01a6400 con 0x7f00c0106d10 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f00c01a3050 con 0x7f00c0106d10 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f00c01a3590 con 0x7f00c0106d10 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f00b0001150 con 0x7f00c0106d10 2024-04-14T15:10:26.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.561+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f00b0004c80 con 0x7f00c0106d10 2024-04-14T15:10:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.562+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a4005150 con 0x7f00c0106d10 2024-04-14T15:10:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.563+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 16) ==== 92674+0+0 (unknown 428266710 0 0) 0x7f00b0001ae0 con 0x7f00c0106d10 2024-04-14T15:10:26.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.563+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f00b0088fe0 con 0x7f00c0106d10 2024-04-14T15:10:26.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.568+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f00b0056160 con 0x7f00c0106d10 2024-04-14T15:10:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:26 smithi071 ceph-mon[33150]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:10:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.861+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f00a4005440 con 0x7f00c0106d10 2024-04-14T15:10:26.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.866+0000 7f00b67fc700 1 -- 172.21.15.45:0/3258445934 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (unknown 57899324 0 3283423196) 0x7f00b0055a60 con 0x7f00c0106d10 2024-04-14T15:10:26.866 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-14T15:10:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQDi8RtmHaZ0MxAA5SeREtjUzQaa9mS6CaFWLQ== 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 >> v1:172.21.15.45:6800/2191709230 conn(0x7f00ac06f960 legacy=0x7f00ac071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 >> v1:172.21.15.71:6789/0 conn(0x7f00c0106d10 legacy=0x7f00c019e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 shutdown_connections 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 >> 172.21.15.45:0/3258445934 conn(0x7f00c00fde80 msgr2=0x7f00c0109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 shutdown_connections 2024-04-14T15:10:26.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:26.868+0000 7f00c7284700 1 -- 172.21.15.45:0/3258445934 wait complete. 2024-04-14T15:10:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3258445934' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:27 smithi071 ceph-mon[33150]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:27 smithi071 ceph-mon[33150]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3258445934' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[29624]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[29624]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3258445934' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[37795]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:27 smithi045 ceph-mon[37795]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:27.585 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:10:27.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-14T15:10:27.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-14T15:10:27.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-14T15:10:28.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:28 smithi071 ceph-mon[33150]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-04-14T15:10:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:28 smithi045 ceph-mon[29624]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-04-14T15:10:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:28 smithi045 ceph-mon[37795]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-04-14T15:10:29.650 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.b/config 2024-04-14T15:10:30.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:30 smithi071 ceph-mon[33150]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:30 smithi045 ceph-mon[29624]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:30 smithi045 ceph-mon[37795]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.348+0000 7f9cca94f700 1 -- 172.21.15.71:0/2614188795 >> v1:172.21.15.71:6789/0 conn(0x7f9cc4108f20 legacy=0x7f9cc410b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.348+0000 7f9cca94f700 1 -- 172.21.15.71:0/2614188795 shutdown_connections 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.348+0000 7f9cca94f700 1 -- 172.21.15.71:0/2614188795 >> 172.21.15.71:0/2614188795 conn(0x7f9cc41000b0 msgr2=0x7f9cc41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.348+0000 7f9cca94f700 1 -- 172.21.15.71:0/2614188795 shutdown_connections 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.348+0000 7f9cca94f700 1 -- 172.21.15.71:0/2614188795 wait complete. 2024-04-14T15:10:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cca94f700 1 Processor -- start 2024-04-14T15:10:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cca94f700 1 -- start start 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cca94f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cc419ac10 con 0x7f9cc410cc60 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cca94f700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cc41a6400 con 0x7f9cc4108f20 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cca94f700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cc41a75e0 con 0x7f9cc4104af0 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cc3fff700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9cc4104af0 0x7f9cc419a0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.71:36092/0 (socket says 172.21.15.71:36092) 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cc37fe700 1 --1- >> v1:172.21.15.71:6789/0 conn(0x7f9cc4108f20 0x7f9cc41a05a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.71:6789/0 says I am v1:172.21.15.71:46362/0 (socket says 172.21.15.71:46362) 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.349+0000 7f9cc3fff700 1 -- 172.21.15.71:0/1991267635 learned_addr learned my addr 172.21.15.71:0/1991267635 (peer_addr_for_me v1:172.21.15.71:0/0) 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567085541 0 0) 0x7f9cc41a75e0 con 0x7f9cc4104af0 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cac0035b0 con 0x7f9cc4104af0 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776671213 0 0) 0x7f9cc419ac10 con 0x7f9cc410cc60 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cc41a75e0 con 0x7f9cc410cc60 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991969069 0 0) 0x7f9cc41a6400 con 0x7f9cc4108f20 2024-04-14T15:10:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cc419ac10 con 0x7f9cc4108f20 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3546530805 0 0) 0x7f9cac0035b0 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cc41a6400 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cb40017f0 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4249592558 0 0) 0x7f9cc41a75e0 con 0x7f9cc410cc60 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cac0035b0 con 0x7f9cc410cc60 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cbc001b70 con 0x7f9cc410cc60 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3663625633 0 0) 0x7f9cc41a6400 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 >> v1:172.21.15.71:6789/0 conn(0x7f9cc4108f20 legacy=0x7f9cc41a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 >> v1:172.21.15.45:6789/0 conn(0x7f9cc410cc60 legacy=0x7f9cc41a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.350+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc41a8800 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.351+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc41a7810 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.351+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9cc41a7d50 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.351+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9cb4003b30 con 0x7f9cc4104af0 2024-04-14T15:10:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.351+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cb4005b80 con 0x7f9cc4104af0 2024-04-14T15:10:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.352+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc40693d0 con 0x7f9cc4104af0 2024-04-14T15:10:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.352+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 16) ==== 92674+0+0 (unknown 428266710 0 0) 0x7f9cb4005b80 con 0x7f9cc4104af0 2024-04-14T15:10:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.353+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f9cb408a0a0 con 0x7f9cc4104af0 2024-04-14T15:10:31.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.358+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9cb4057200 con 0x7f9cc4104af0 2024-04-14T15:10:31.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.647+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9cc4111fc0 con 0x7f9cc4104af0 2024-04-14T15:10:31.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.653+0000 7f9cc17fa700 1 -- 172.21.15.71:0/1991267635 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) ==== 170+0+59 (unknown 3508017507 0 1588948646) 0x7f9cb4056b00 con 0x7f9cc4104af0 2024-04-14T15:10:31.654 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2024-04-14T15:10:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDn8Rtmodu8JhAA9ouUwdopxEvLFD64Wi4mtg== 2024-04-14T15:10:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9cac06f7d0 legacy=0x7f9cac071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 >> v1:172.21.15.45:6790/0 conn(0x7f9cc4104af0 legacy=0x7f9cc419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 shutdown_connections 2024-04-14T15:10:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 >> 172.21.15.71:0/1991267635 conn(0x7f9cc41000b0 msgr2=0x7f9cc410b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 shutdown_connections 2024-04-14T15:10:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-14T15:10:31.655+0000 7f9cca94f700 1 -- 172.21.15.71:0/1991267635 wait complete. 2024-04-14T15:10:32.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:10:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:10:31.773Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-14T15:10:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[37795]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.71:0/1991267635' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[37795]: from='client.34484 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[37795]: from='client.34484 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[29624]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.71:0/1991267635' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[29624]: from='client.34484 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:32 smithi045 ceph-mon[29624]: from='client.34484 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:32.585 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:32 smithi071 ceph-mon[33150]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:10:32.585 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.71:0/1991267635' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.585 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:32 smithi071 ceph-mon[33150]: from='client.34484 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-14T15:10:32.585 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:32 smithi071 ceph-mon[33150]: from='client.34484 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-14T15:10:32.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:10:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:10:34.275 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-14T15:10:34.275 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-14T15:10:34.275 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-14T15:10:34.301 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:10:34 smithi071 bash[61782]: 3c7d7117f239f521fd96dbb46007788eb6f433567109d77b8b7048ea8d362179 2024-04-14T15:10:34.301 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:10:34 smithi071 systemd[1]: Started Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:34 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-14T15:10:34.319 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-14T15:10:34.319 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-14T15:10:34.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph mgr dump --format=json 2024-04-14T15:10:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' 2024-04-14T15:10:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:34 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-14T15:10:34.577 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:35.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setuser ceph since I am not root 2024-04-14T15:10:35.246 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ignoring --setgroup ceph since I am not root 2024-04-14T15:10:35.607 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: ignoring --setuser ceph since I am not root 2024-04-14T15:10:35.607 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: ignoring --setgroup ceph since I am not root 2024-04-14T15:10:35.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:35.733+0000 7f602491d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:10:35.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:35 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:35.836+0000 7f602491d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:10:36.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:35.716+0000 7f8ed94c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-14T15:10:36.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:35.809+0000 7f8ed94c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-14T15:10:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.277+0000 7f9cbee5d700 1 -- 172.21.15.45:0/1261625762 >> v1:172.21.15.45:6789/0 conn(0x7f9cb810f2f0 legacy=0x7f9cb81117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.277+0000 7f9cbee5d700 1 -- 172.21.15.45:0/1261625762 shutdown_connections 2024-04-14T15:10:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.277+0000 7f9cbee5d700 1 -- 172.21.15.45:0/1261625762 >> 172.21.15.45:0/1261625762 conn(0x7f9cb80fe3a0 msgr2=0x7f9cb81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:36.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.277+0000 7f9cbee5d700 1 -- 172.21.15.45:0/1261625762 shutdown_connections 2024-04-14T15:10:36.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.277+0000 7f9cbee5d700 1 -- 172.21.15.45:0/1261625762 wait complete. 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbee5d700 1 Processor -- start 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbee5d700 1 -- start start 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbee5d700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cb810ef40 con 0x7f9cb810b720 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbee5d700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cb81a8f00 con 0x7f9cb810f2f0 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbee5d700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9cb81aa0e0 con 0x7f9cb8108870 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbcbf9700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9cb8108870 0x7f9cb810d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55648/0 (socket says 172.21.15.45:55648) 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cbcbf9700 1 -- 172.21.15.45:0/3880695799 learned_addr learned my addr 172.21.15.45:0/3880695799 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001002765 0 0) 0x7f9cb810ef40 con 0x7f9cb810b720 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca80035b0 con 0x7f9cb810b720 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306475262 0 0) 0x7f9cb81aa0e0 con 0x7f9cb8108870 2024-04-14T15:10:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb810ef40 con 0x7f9cb8108870 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553192904 0 0) 0x7f9cb81a8f00 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.278+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cb81aa0e0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2698826553 0 0) 0x7f9cb81aa0e0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cb81a8f00 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cb00017b0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1969627434 0 0) 0x7f9cb810ef40 con 0x7f9cb8108870 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cb81aa0e0 con 0x7f9cb8108870 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4285886905 0 0) 0x7f9ca80035b0 con 0x7f9cb810b720 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9cb810ef40 con 0x7f9cb810b720 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cac0017f0 con 0x7f9cb8108870 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ca4001830 con 0x7f9cb810b720 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3414054715 0 0) 0x7f9cb81a8f00 con 0x7f9cb810f2f0 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.45:6790/0 conn(0x7f9cb8108870 legacy=0x7f9cb810d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.45:6789/0 conn(0x7f9cb810b720 legacy=0x7f9cb810df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:36.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81ab2c0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81a7f50 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9cb0001a10 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9cb0005110 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.279+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9cb81a8410 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.280+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb8103fb0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.281+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 17) ==== 92688+0+0 (unknown 4162667236 0 0) 0x7f9cb001bd90 con 0x7f9cb810f2f0 2024-04-14T15:10:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.281+0000 7f9cbcbf9700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9ca806fb70 legacy=0x7f9ca8072030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.281+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f9cb00893c0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.286+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9cb0056520 con 0x7f9cb810f2f0 2024-04-14T15:10:36.451 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-mon[33150]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-14T15:10:36.452 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-mon[33150]: mgrmap e17: y(active, since 4m), standbys: x 2024-04-14T15:10:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.482+0000 7f9cbcbf9700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9ca806fb70 legacy=0x7f9ca8072030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:36.482 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-mon[37795]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-14T15:10:36.482 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-mon[37795]: mgrmap e17: y(active, since 4m), standbys: x 2024-04-14T15:10:36.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-mon[29624]: from='mgr.14152 v1:172.21.15.45:0/3330400389' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-14T15:10:36.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-mon[29624]: mgrmap e17: y(active, since 4m), standbys: x 2024-04-14T15:10:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.551+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9cb810d2d0 con 0x7f9cb810f2f0 2024-04-14T15:10:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.554+0000 7f9cb5ffb700 1 -- 172.21.15.45:0/3880695799 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+177153 (unknown 831476310 0 2473197015) 0x7f9cb0055e20 con 0x7f9cb810f2f0 2024-04-14T15:10:36.557 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:10:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.557+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9ca806fb70 legacy=0x7f9ca8072030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:10:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.557+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 >> v1:172.21.15.71:6789/0 conn(0x7f9cb810f2f0 legacy=0x7f9cb81a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.558+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 shutdown_connections 2024-04-14T15:10:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.558+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 >> 172.21.15.45:0/3880695799 conn(0x7f9cb80fe3a0 msgr2=0x7f9cb8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.558+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 shutdown_connections 2024-04-14T15:10:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:36.558+0000 7f9cbee5d700 1 -- 172.21.15.45:0/3880695799 wait complete. 2024-04-14T15:10:36.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:36.513+0000 7f602491d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:10:36.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:36.613+0000 7f602491d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:10:36.781 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:36.450+0000 7f8ed94c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-14T15:10:36.781 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:36.543+0000 7f8ed94c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-14T15:10:36.782 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:36.685+0000 7f8ed94c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:10:36.782 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:36.780+0000 7f8ed94c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:10:37.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:36.864+0000 7f8ed94c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:10:37.152 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:36.758+0000 7f602491d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-14T15:10:37.152 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:36.859+0000 7f602491d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-14T15:10:37.152 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:36.944+0000 7f602491d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-14T15:10:37.210 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6800","nonce":2191709230}]},"active_addr":"172.21.15.45:6800/2191709230","active_change":"2024-04-14T15:05:38.510405+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2385056690}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3124650148}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":558376632}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2492949599}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":405902173}]}]} 2024-04-14T15:10:37.214 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-14T15:10:37.214 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-14T15:10:37.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd dump --format=json 2024-04-14T15:10:37.423 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3880695799' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-14T15:10:37.423 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:37.127+0000 7f8ed94c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:10:37.423 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:37.237+0000 7f8ed94c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:10:37.423 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:37.326+0000 7f8ed94c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:10:37.423 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:37.421+0000 7f8ed94c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:10:37.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3880695799' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-14T15:10:37.431 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3880695799' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-14T15:10:37.431 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:37.218+0000 7f602491d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-14T15:10:37.431 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:37.335+0000 7f602491d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-14T15:10:37.466 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:37.707 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:37.430+0000 7f602491d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-14T15:10:37.707 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:37.536+0000 7f602491d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-14T15:10:37.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:37.598+0000 7f8ed94c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:10:37.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:37 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:37.722+0000 7f602491d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-14T15:10:38.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:38.548+0000 7f8ed94c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:10:38.857 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:38.678+0000 7f8ed94c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:10:38.869 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:38.726+0000 7f602491d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-14T15:10:38.869 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:38.869+0000 7f602491d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-14T15:10:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.950+0000 7f4a46cdf700 1 -- 172.21.15.45:0/660555481 >> v1:172.21.15.45:6789/0 conn(0x7f4a4010b890 legacy=0x7f4a4010dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.950+0000 7f4a46cdf700 1 -- 172.21.15.45:0/660555481 shutdown_connections 2024-04-14T15:10:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.950+0000 7f4a46cdf700 1 -- 172.21.15.45:0/660555481 >> 172.21.15.45:0/660555481 conn(0x7f4a400fd290 msgr2=0x7f4a400ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.950+0000 7f4a46cdf700 1 -- 172.21.15.45:0/660555481 shutdown_connections 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.950+0000 7f4a46cdf700 1 -- 172.21.15.45:0/660555481 wait complete. 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a46cdf700 1 Processor -- start 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a46cdf700 1 -- start start 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a46cdf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4a401a7850 con 0x7f4a4010f3a0 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a46cdf700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4a401a8a70 con 0x7f4a4010b890 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a46cdf700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f4a401a9cb0 con 0x7f4a40103a80 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a44a7b700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f4a40103a80 0x7f4a4010afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:55660/0 (socket says 172.21.15.45:55660) 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a4527c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4a4010f3a0 0x7f4a401a5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46826/0 (socket says 172.21.15.45:46826) 2024-04-14T15:10:38.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.951+0000 7f4a4527c700 1 -- 172.21.15.45:0/581267002 learned_addr learned my addr 172.21.15.45:0/581267002 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271161549 0 0) 0x7f4a401a7850 con 0x7f4a4010f3a0 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a300035b0 con 0x7f4a4010f3a0 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2186828424 0 0) 0x7f4a401a9cb0 con 0x7f4a40103a80 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a401a7850 con 0x7f4a40103a80 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412399992 0 0) 0x7f4a401a8a70 con 0x7f4a4010b890 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a401a9cb0 con 0x7f4a4010b890 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1337437550 0 0) 0x7f4a300035b0 con 0x7f4a4010f3a0 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4a401a8a70 con 0x7f4a4010f3a0 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4027036707 0 0) 0x7f4a401a9cb0 con 0x7f4a4010b890 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4a300035b0 con 0x7f4a4010b890 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f4a38001c50 con 0x7f4a4010f3a0 2024-04-14T15:10:38.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1558183360 0 0) 0x7f4a401a7850 con 0x7f4a40103a80 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4a401a9cb0 con 0x7f4a40103a80 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f4a2c004460 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f4a340012d0 con 0x7f4a40103a80 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3808569751 0 0) 0x7f4a300035b0 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.45:6790/0 conn(0x7f4a40103a80 legacy=0x7f4a4010afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.45:6789/0 conn(0x7f4a4010f3a0 legacy=0x7f4a401a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a401aaef0 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4a401a9ee0 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f4a2c001af0 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f4a2c004f20 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.952+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4a401aa3a0 con 0x7f4a4010b890 2024-04-14T15:10:38.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.953+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4006ace0 con 0x7f4a4010b890 2024-04-14T15:10:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.954+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 17) ==== 92688+0+0 (unknown 4162667236 0 0) 0x7f4a2c003860 con 0x7f4a4010b890 2024-04-14T15:10:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.955+0000 7f4a44a7b700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.45:6800/2191709230 conn(0x7f4a3006f9b0 legacy=0x7f4a30071e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:38.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.955+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f4a2c0891c0 con 0x7f4a4010b890 2024-04-14T15:10:38.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:38.959+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f4a2c056320 con 0x7f4a4010b890 2024-04-14T15:10:39.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.156+0000 7f4a44a7b700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.45:6800/2191709230 conn(0x7f4a3006f9b0 legacy=0x7f4a30071e70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:39.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.168+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a40069400 con 0x7f4a4010b890 2024-04-14T15:10:39.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.169+0000 7f4a3dffb700 1 -- 172.21.15.45:0/581267002 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+20550 (unknown 2171855796 0 3020125770) 0x7f4a40069400 con 0x7f4a4010b890 2024-04-14T15:10:39.170 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:10:39.170 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","created":"2024-04-14T15:04:40.069266+0000","modified":"2024-04-14T15:10:14.929856+0000","last_up_change":"2024-04-14T15:09:48.865155+0000","last_in_change":"2024-04-14T15:09:30.892466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T15:07:48.583816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-04-14T15:09:59.937453+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-04-14T15:10:00.076847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-04-14T15:10:01.349923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-04-14T15:10:03.308695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-04-14T15:10:05.381681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ceb62cce-7685-4b96-930f-bba7be80eabd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":609845118}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":609845118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":609845118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":609845118}]},"public_addr":"172.21.15.45:6801/609845118","cluster_addr":"172.21.15.45:6802/609845118","heartbeat_back_addr":"172.21.15.45:6804/609845118","heartbeat_front_addr":"172.21.15.45:6803/609845118","state":["exists","up"]},{"osd":1,"uuid":"39c5f730-a033-4475-9c37-37fe6929650d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":1698650141}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":1698650141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":1698650141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":1698650141}]},"public_addr":"172.21.15.45:6805/1698650141","cluster_addr":"172.21.15.45:6806/1698650141","heartbeat_back_addr":"172.21.15.45:6808/1698650141","heartbeat_front_addr":"172.21.15.45:6807/1698650141","state":["exists","up"]},{"osd":2,"uuid":"11bba3d9-26dc-4989-ae9d-555847390c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":1422326616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":1422326616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6812","nonce":1422326616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":1422326616}]},"public_addr":"172.21.15.45:6809/1422326616","cluster_addr":"172.21.15.45:6810/1422326616","heartbeat_back_addr":"172.21.15.45:6812/1422326616","heartbeat_front_addr":"172.21.15.45:6811/1422326616","state":["exists","up"]},{"osd":3,"uuid":"16ce0a73-cedd-4ea0-b121-f9900c96b0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6813","nonce":2647051701}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6814","nonce":2647051701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6816","nonce":2647051701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6815","nonce":2647051701}]},"public_addr":"172.21.15.45:6813/2647051701","cluster_addr":"172.21.15.45:6814/2647051701","heartbeat_back_addr":"172.21.15.45:6816/2647051701","heartbeat_front_addr":"172.21.15.45:6815/2647051701","state":["exists","up"]},{"osd":4,"uuid":"35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6800","nonce":2605280975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":2605280975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":2605280975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":2605280975}]},"public_addr":"172.21.15.71:6800/2605280975","cluster_addr":"172.21.15.71:6801/2605280975","heartbeat_back_addr":"172.21.15.71:6803/2605280975","heartbeat_front_addr":"172.21.15.71:6802/2605280975","state":["exists","up"]},{"osd":5,"uuid":"db3f2e72-cdfb-431f-a7e0-25f137f1d3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":3293828982}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":3293828982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":3293828982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":3293828982}]},"public_addr":"172.21.15.71:6804/3293828982","cluster_addr":"172.21.15.71:6805/3293828982","heartbeat_back_addr":"172.21.15.71:6807/3293828982","heartbeat_front_addr":"172.21.15.71:6806/3293828982","state":["exists","up"]},{"osd":6,"uuid":"d2c28b0b-6337-4095-bc52-631380978e4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":1175230288}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":1175230288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":1175230288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":1175230288}]},"public_addr":"172.21.15.71:6808/1175230288","cluster_addr":"172.21.15.71:6809/1175230288","heartbeat_back_addr":"172.21.15.71:6811/1175230288","heartbeat_front_addr":"172.21.15.71:6810/1175230288","state":["exists","up"]},{"osd":7,"uuid":"1374c765-77e1-49d7-88b5-cbb13810ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":2779299537}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":2779299537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":2779299537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":2779299537}]},"public_addr":"172.21.15.71:6812/2779299537","cluster_addr":"172.21.15.71:6813/2779299537","heartbeat_back_addr":"172.21.15.71:6815/2779299537","heartbeat_front_addr":"172.21.15.71:6814/2779299537","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:06:56.929871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:20.714342+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:45.708756+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:10.267994+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:32.921703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:57.376518+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:22.153259+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:46.961704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4130748887":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/2960091533":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/3894205139":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/2089996418":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/3535859446":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/2598686815":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/3911040716":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/1260581702":"2024-04-15T15:05:38.510265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.45:6800/2191709230 conn(0x7f4a3006f9b0 legacy=0x7f4a30071e70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 >> v1:172.21.15.71:6789/0 conn(0x7f4a4010b890 legacy=0x7f4a401a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 shutdown_connections 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 >> 172.21.15.45:0/581267002 conn(0x7f4a400fd290 msgr2=0x7f4a400ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 shutdown_connections 2024-04-14T15:10:39.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:39.172+0000 7f4a46cdf700 1 -- 172.21.15.45:0/581267002 wait complete. 2024-04-14T15:10:39.297 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/581267002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:39.297 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:39.008+0000 7f8ed94c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:10:39.297 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:39.097+0000 7f8ed94c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:10:39.448 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/581267002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:39.448 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/581267002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:39.448 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:39.222+0000 7f602491d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-14T15:10:39.448 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:39.316+0000 7f602491d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-14T15:10:39.607 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:39.295+0000 7f8ed94c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:10:39.607 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:39.483+0000 7f8ed94c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:10:39.716 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:39.523+0000 7f602491d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:10:39.860 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-14T15:10:39.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd dump --format=json 2024-04-14T15:10:39.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:39 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:39.715+0000 7f602491d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-14T15:10:40.026 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:39.768+0000 7f8ed94c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:10:40.110 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:40.291 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:40 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:40.019+0000 7f602491d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-14T15:10:40.357 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:40.025+0000 7f8ed94c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:10:40.357 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:40.261+0000 7f8ed94c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:10:40.545 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:40 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:40.291+0000 7f602491d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-14T15:10:40.546 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:40 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:40.544+0000 7f602491d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-14T15:10:41.357 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:41.060+0000 7f8ed94c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:10:41.602 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:41 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:41.388+0000 7f602491d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-14T15:10:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.860+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1166217299 >> v1:172.21.15.45:6789/0 conn(0x7f9f041078e0 legacy=0x7f9f04107cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:41.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.860+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1166217299 shutdown_connections 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.860+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1166217299 >> 172.21.15.45:0/1166217299 conn(0x7f9f040fd2c0 msgr2=0x7f9f040ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.861+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1166217299 shutdown_connections 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.861+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1166217299 wait complete. 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.861+0000 7f9f0bc84700 1 Processor -- start 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.861+0000 7f9f0bc84700 1 -- start start 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.861+0000 7f9f0bc84700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f041a5a90 con 0x7f9f041078e0 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f0bc84700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f041a6cb0 con 0x7f9f0410e360 2024-04-14T15:10:41.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f0bc84700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9f041a6ee0 con 0x7f9f0410a790 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f0921f700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9f0410a790 0x7f9f041a0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45054/0 (socket says 172.21.15.45:45054) 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f09a20700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f041078e0 0x7f9f0410dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37080/0 (socket says 172.21.15.45:37080) 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f0921f700 1 -- 172.21.15.45:0/1489280856 learned_addr learned my addr 172.21.15.45:0/1489280856 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133964074 0 0) 0x7f9f041a5a90 con 0x7f9f041078e0 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eec0035b0 con 0x7f9f041078e0 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850166553 0 0) 0x7f9f041a6ee0 con 0x7f9f0410a790 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f041a5a90 con 0x7f9f0410a790 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202493578 0 0) 0x7f9f041a6cb0 con 0x7f9f0410e360 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f041a6ee0 con 0x7f9f0410e360 2024-04-14T15:10:41.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 90017160 0 0) 0x7f9f041a5a90 con 0x7f9f0410a790 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9f041a6cb0 con 0x7f9f0410a790 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ef80018a0 con 0x7f9f0410a790 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2963320922 0 0) 0x7f9f041a6ee0 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9f041a5a90 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.862+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ef4001bb0 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 838550198 0 0) 0x7f9f041a5a90 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.45:6790/0 conn(0x7f9f0410a790 legacy=0x7f9f041a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.45:6789/0 conn(0x7f9f041078e0 legacy=0x7f9f0410dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f041a7110 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9f041a5cc0 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9f041a6180 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9ef4003eb0 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.863+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ef4004d40 con 0x7f9f0410e360 2024-04-14T15:10:41.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.864+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef0005120 con 0x7f9f0410e360 2024-04-14T15:10:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.864+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 17) ==== 92688+0+0 (unknown 4162667236 0 0) 0x7f9ef4003a20 con 0x7f9f0410e360 2024-04-14T15:10:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.865+0000 7f9f09a20700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9eec06f7a0 legacy=0x7f9eec071c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:41.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.865+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(69..69 src has 1..69) ==== 5942+0+0 (unknown 1502571632 0 0) 0x7f9ef4089290 con 0x7f9f0410e360 2024-04-14T15:10:41.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:41.869+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9ef40563f0 con 0x7f9f0410e360 2024-04-14T15:10:41.951 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:41.632+0000 7f8ed94c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:10:41.951 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:41.731+0000 7f8ed94c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:10:42.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.010+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 10 ==== mgrmap(e 18) ==== 92688+0+0 (unknown 3184842240 0 0) 0x7f9ef40519b0 con 0x7f9f0410e360 2024-04-14T15:10:42.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.066+0000 7f9f09a20700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9eec06f7a0 legacy=0x7f9eec071c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.45:6800/2191709230 2024-04-14T15:10:42.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.081+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ef0005410 con 0x7f9f0410e360 2024-04-14T15:10:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.083+0000 7f9f02ffd700 1 -- 172.21.15.45:0/1489280856 <== mon.1 v1:172.21.15.71:6789/0 11 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+20550 (unknown 2171855796 0 3020125770) 0x7f9ef4068330 con 0x7f9f0410e360 2024-04-14T15:10:42.084 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:10:42.084 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":69,"fsid":"1eba33fc-fa70-11ee-bc8e-c7b262605968","created":"2024-04-14T15:04:40.069266+0000","modified":"2024-04-14T15:10:14.929856+0000","last_up_change":"2024-04-14T15:09:48.865155+0000","last_in_change":"2024-04-14T15:09:30.892466+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-14T15:07:48.583816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-04-14T15:09:59.937453+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-04-14T15:10:00.076847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-04-14T15:10:01.349923+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-04-14T15:10:03.308695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-04-14T15:10:05.381681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ceb62cce-7685-4b96-930f-bba7be80eabd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6801","nonce":609845118}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6802","nonce":609845118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6804","nonce":609845118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6803","nonce":609845118}]},"public_addr":"172.21.15.45:6801/609845118","cluster_addr":"172.21.15.45:6802/609845118","heartbeat_back_addr":"172.21.15.45:6804/609845118","heartbeat_front_addr":"172.21.15.45:6803/609845118","state":["exists","up"]},{"osd":1,"uuid":"39c5f730-a033-4475-9c37-37fe6929650d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6805","nonce":1698650141}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6806","nonce":1698650141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6808","nonce":1698650141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6807","nonce":1698650141}]},"public_addr":"172.21.15.45:6805/1698650141","cluster_addr":"172.21.15.45:6806/1698650141","heartbeat_back_addr":"172.21.15.45:6808/1698650141","heartbeat_front_addr":"172.21.15.45:6807/1698650141","state":["exists","up"]},{"osd":2,"uuid":"11bba3d9-26dc-4989-ae9d-555847390c3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6809","nonce":1422326616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6810","nonce":1422326616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6812","nonce":1422326616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6811","nonce":1422326616}]},"public_addr":"172.21.15.45:6809/1422326616","cluster_addr":"172.21.15.45:6810/1422326616","heartbeat_back_addr":"172.21.15.45:6812/1422326616","heartbeat_front_addr":"172.21.15.45:6811/1422326616","state":["exists","up"]},{"osd":3,"uuid":"16ce0a73-cedd-4ea0-b121-f9900c96b0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6813","nonce":2647051701}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6814","nonce":2647051701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6816","nonce":2647051701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.45:6815","nonce":2647051701}]},"public_addr":"172.21.15.45:6813/2647051701","cluster_addr":"172.21.15.45:6814/2647051701","heartbeat_back_addr":"172.21.15.45:6816/2647051701","heartbeat_front_addr":"172.21.15.45:6815/2647051701","state":["exists","up"]},{"osd":4,"uuid":"35bfb6b8-5bb1-446c-ab20-65b2e1b0ba0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6800","nonce":2605280975}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":2605280975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":2605280975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":2605280975}]},"public_addr":"172.21.15.71:6800/2605280975","cluster_addr":"172.21.15.71:6801/2605280975","heartbeat_back_addr":"172.21.15.71:6803/2605280975","heartbeat_front_addr":"172.21.15.71:6802/2605280975","state":["exists","up"]},{"osd":5,"uuid":"db3f2e72-cdfb-431f-a7e0-25f137f1d3de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":3293828982}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":3293828982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":3293828982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":3293828982}]},"public_addr":"172.21.15.71:6804/3293828982","cluster_addr":"172.21.15.71:6805/3293828982","heartbeat_back_addr":"172.21.15.71:6807/3293828982","heartbeat_front_addr":"172.21.15.71:6806/3293828982","state":["exists","up"]},{"osd":6,"uuid":"d2c28b0b-6337-4095-bc52-631380978e4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":1175230288}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":1175230288}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":1175230288}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":1175230288}]},"public_addr":"172.21.15.71:6808/1175230288","cluster_addr":"172.21.15.71:6809/1175230288","heartbeat_back_addr":"172.21.15.71:6811/1175230288","heartbeat_front_addr":"172.21.15.71:6810/1175230288","state":["exists","up"]},{"osd":7,"uuid":"1374c765-77e1-49d7-88b5-cbb13810ef94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":2779299537}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":2779299537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":2779299537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":2779299537}]},"public_addr":"172.21.15.71:6812/2779299537","cluster_addr":"172.21.15.71:6813/2779299537","heartbeat_back_addr":"172.21.15.71:6815/2779299537","heartbeat_front_addr":"172.21.15.71:6814/2779299537","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:06:56.929871+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:20.714342+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:07:45.708756+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:10.267994+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:32.921703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:08:57.376518+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:22.153259+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-14T15:09:46.961704+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4130748887":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/2960091533":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/3894205139":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/2089996418":"2024-04-15T15:05:12.202431+0000","172.21.15.45:0/3535859446":"2024-04-15T15:05:12.202431+0000","172.21.15.45:6800/2598686815":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/3911040716":"2024-04-15T15:05:38.510265+0000","172.21.15.45:0/1260581702":"2024-04-15T15:05:38.510265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-14T15:10:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.085+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.45:6800/2191709230 conn(0x7f9eec06f7a0 legacy=0x7f9eec071c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-14T15:10:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.085+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 >> v1:172.21.15.71:6789/0 conn(0x7f9f0410e360 legacy=0x7f9f041a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.085+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 shutdown_connections 2024-04-14T15:10:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.085+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 >> 172.21.15.45:0/1489280856 conn(0x7f9f040fd2c0 msgr2=0x7f9f0406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.086+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 shutdown_connections 2024-04-14T15:10:42.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:42.086+0000 7f9f0bc84700 1 -- 172.21.15.45:0/1489280856 wait complete. 2024-04-14T15:10:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: Standby manager daemon x restarted 2024-04-14T15:10:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: Standby manager daemon x started 2024-04-14T15:10:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:10:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[37795]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: Standby manager daemon x restarted 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: Standby manager daemon x started 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-mon[29624]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:10:42.247 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:41 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:41.993+0000 7f602491d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-14T15:10:42.248 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:42.101+0000 7f602491d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-14T15:10:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: Standby manager daemon x restarted 2024-04-14T15:10:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: Standby manager daemon x started 2024-04-14T15:10:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-04-14T15:10:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-14T15:10:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-mon[33150]: from='mgr.? v1:172.21.15.71:0/3506790306' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:41.949+0000 7f8ed94c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: [14/Apr/2024:15:10:41] ENGINE Bus STARTING 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: CherryPy Checker: 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: The Application mounted at '' has an empty config. 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: [14/Apr/2024:15:10:42] ENGINE Serving on http://:::9283 2024-04-14T15:10:42.358 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:10:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x[35001]: [14/Apr/2024:15:10:42] ENGINE Bus STARTED 2024-04-14T15:10:42.529 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:42.330+0000 7f602491d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-14T15:10:42.529 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:10:42] ENGINE Bus STARTING 2024-04-14T15:10:42.803 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: CherryPy Checker: 2024-04-14T15:10:42.803 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: The Application mounted at '' has an empty config. 2024-04-14T15:10:42.803 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:10:42.804 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:10:42] ENGINE Serving on http://:::9283 2024-04-14T15:10:42.804 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:42 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:10:42] ENGINE Bus STARTED 2024-04-14T15:10:42.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-14T15:10:42.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-14T15:10:42.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-14T15:10:42.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-14T15:10:42.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-14T15:10:42.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-14T15:10:42.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-14T15:10:42.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: mgrmap e18: y(active, since 5m), standbys: x 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1489280856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: Active manager daemon y restarted 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: Activating manager daemon y 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: osdmap e70: 8 total, 8 up, 8 in 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: mgrmap e19: y(active, starting, since 0.00734467s), standbys: x 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:10:43.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: Manager daemon y is now available 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: mgrmap e18: y(active, since 5m), standbys: x 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1489280856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: Active manager daemon y restarted 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: Activating manager daemon y 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: osdmap e70: 8 total, 8 up, 8 in 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: mgrmap e19: y(active, starting, since 0.00734467s), standbys: x 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:10:43.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:10:43.251 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: Manager daemon y is now available 2024-04-14T15:10:43.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:43.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:43.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:10:43.252 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:10:43.333 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: mgrmap e18: y(active, since 5m), standbys: x 2024-04-14T15:10:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1489280856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: Active manager daemon y restarted 2024-04-14T15:10:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: Activating manager daemon y 2024-04-14T15:10:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: osdmap e70: 8 total, 8 up, 8 in 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: mgrmap e19: y(active, starting, since 0.00734467s), standbys: x 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-14T15:10:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: Manager daemon y is now available 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-04-14T15:10:43.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-04-14T15:10:43.375 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.409 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.647 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.648 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:43.655 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:10:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:10:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[37795]: mgrmap e20: y(active, since 1.03643s), standbys: x 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[37795]: [14/Apr/2024:15:10:43] ENGINE Bus STARTING 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[37795]: [14/Apr/2024:15:10:44] ENGINE Serving on http://172.21.15.45:8765 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[37795]: [14/Apr/2024:15:10:44] ENGINE Serving on https://172.21.15.45:7150 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[37795]: [14/Apr/2024:15:10:44] ENGINE Bus STARTED 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[29624]: mgrmap e20: y(active, since 1.03643s), standbys: x 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[29624]: [14/Apr/2024:15:10:43] ENGINE Bus STARTING 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[29624]: [14/Apr/2024:15:10:44] ENGINE Serving on http://172.21.15.45:8765 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[29624]: [14/Apr/2024:15:10:44] ENGINE Serving on https://172.21.15.45:7150 2024-04-14T15:10:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:44 smithi045 ceph-mon[29624]: [14/Apr/2024:15:10:44] ENGINE Bus STARTED 2024-04-14T15:10:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:44 smithi071 ceph-mon[33150]: mgrmap e20: y(active, since 1.03643s), standbys: x 2024-04-14T15:10:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:44 smithi071 ceph-mon[33150]: [14/Apr/2024:15:10:43] ENGINE Bus STARTING 2024-04-14T15:10:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:44 smithi071 ceph-mon[33150]: [14/Apr/2024:15:10:44] ENGINE Serving on http://172.21.15.45:8765 2024-04-14T15:10:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:44 smithi071 ceph-mon[33150]: [14/Apr/2024:15:10:44] ENGINE Serving on https://172.21.15.45:7150 2024-04-14T15:10:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:44 smithi071 ceph-mon[33150]: [14/Apr/2024:15:10:44] ENGINE Bus STARTED 2024-04-14T15:10:45.249 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:10:45.437 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:45 smithi045 ceph-mon[37795]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:45.437 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:45 smithi045 ceph-mon[37795]: mgrmap e21: y(active, since 2s), standbys: x 2024-04-14T15:10:45.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:45 smithi045 ceph-mon[29624]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:45.437 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:45 smithi045 ceph-mon[29624]: mgrmap e21: y(active, since 2s), standbys: x 2024-04-14T15:10:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:45 smithi071 ceph-mon[33150]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:45 smithi071 ceph-mon[33150]: mgrmap e21: y(active, since 2s), standbys: x 2024-04-14T15:10:46.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:46 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:46 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.666 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:46 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.666 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:46 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:46 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:46 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:10:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: mgrmap e22: y(active, since 4s), standbys: x 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 3760M 2024-04-14T15:10:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: mgrmap e22: y(active, since 4s), standbys: x 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 3760M 2024-04-14T15:10:47.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: mgrmap e22: y(active, since 4s), standbys: x 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:10:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 3760M 2024-04-14T15:10:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.145+0000 7fedc633e700 1 -- 172.21.15.45:0/2165147033 >> v1:172.21.15.45:6790/0 conn(0x7fedc010a610 legacy=0x7fedc010ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.145+0000 7fedc633e700 1 -- 172.21.15.45:0/2165147033 shutdown_connections 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.145+0000 7fedc633e700 1 -- 172.21.15.45:0/2165147033 >> 172.21.15.45:0/2165147033 conn(0x7fedc00fd290 msgr2=0x7fedc00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.145+0000 7fedc633e700 1 -- 172.21.15.45:0/2165147033 shutdown_connections 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.145+0000 7fedc633e700 1 -- 172.21.15.45:0/2165147033 wait complete. 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc633e700 1 Processor -- start 2024-04-14T15:10:48.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc633e700 1 -- start start 2024-04-14T15:10:48.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc633e700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fedc01a78e0 con 0x7fedc010a610 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc633e700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fedc01a8b00 con 0x7fedc0107760 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc633e700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fedc01a9d40 con 0x7fedc010e1e0 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedbf7fe700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fedc010a610 0x7fedc01a26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37184/0 (socket says 172.21.15.45:37184) 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedc48db700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fedc010e1e0 0x7fedc01a5fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:45164/0 (socket says 172.21.15.45:45164) 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedbf7fe700 1 -- 172.21.15.45:0/2582927347 learned_addr learned my addr 172.21.15.45:0/2582927347 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353284429 0 0) 0x7fedc01a78e0 con 0x7fedc010a610 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedac0031f0 con 0x7fedc010a610 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.146+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119541554 0 0) 0x7fedc01a9d40 con 0x7fedc010e1e0 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedc01a78e0 con 0x7fedc010e1e0 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2150333564 0 0) 0x7fedc01a8b00 con 0x7fedc0107760 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedc01a9d40 con 0x7fedc0107760 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 796944671 0 0) 0x7fedac0031f0 con 0x7fedc010a610 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedc01a8b00 con 0x7fedc010a610 2024-04-14T15:10:48.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fedb0004500 con 0x7fedc010a610 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 481972329 0 0) 0x7fedc01a78e0 con 0x7fedc010e1e0 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedac0031f0 con 0x7fedc010e1e0 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fedb80015b0 con 0x7fedc010e1e0 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1906351367 0 0) 0x7fedc01a9d40 con 0x7fedc0107760 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fedc01a78e0 con 0x7fedc0107760 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7feda8001830 con 0x7fedc0107760 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3754064434 0 0) 0x7fedc01a8b00 con 0x7fedc010a610 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 >> v1:172.21.15.45:6790/0 conn(0x7fedc010e1e0 legacy=0x7fedc01a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 >> v1:172.21.15.71:6789/0 conn(0x7fedc0107760 legacy=0x7fedc010d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc01aaf80 con 0x7fedc010a610 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fedc01a9f70 con 0x7fedc010a610 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fedc01aa430 con 0x7fedc010a610 2024-04-14T15:10:48.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fedb0001b30 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.147+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fedb0004f30 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.148+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7feda0000fb0 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.149+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fedb00051b0 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.149+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7fedb008a290 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.149+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6809/1422326616 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedac07dde0 con 0x7fedac07a130 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.149+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7fedb008a6d0 con 0x7fedc010a610 2024-04-14T15:10:48.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.150+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== osd.2 v1:172.21.15.45:6809/1422326616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fedac07dde0 con 0x7fedac07a130 2024-04-14T15:10:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.166+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 --> v1:172.21.15.45:6809/1422326616 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feda0002d10 con 0x7fedac07a130 2024-04-14T15:10:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedbd7fa700 1 -- 172.21.15.45:0/2582927347 <== osd.2 v1:172.21.15.45:6809/1422326616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 2210109781) 0x7feda0002d10 con 0x7fedac07a130 2024-04-14T15:10:48.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 >> v1:172.21.15.45:6809/1422326616 conn(0x7fedac07a130 legacy=0x7fedac07c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 >> v1:172.21.15.45:6800/1718216366 conn(0x7fedac06fa30 legacy=0x7fedac071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 >> v1:172.21.15.45:6789/0 conn(0x7fedc010a610 legacy=0x7fedc01a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc48db700 1 -- 172.21.15.45:0/2582927347 reap_dead start 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 shutdown_connections 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 >> 172.21.15.45:0/2582927347 conn(0x7fedc00fd290 msgr2=0x7fedc0110e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 shutdown_connections 2024-04-14T15:10:48.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:48.167+0000 7fedc633e700 1 -- 172.21.15.45:0/2582927347 wait complete. 2024-04-14T15:10:49.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:49 smithi045 ceph-mon[29624]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:49.686 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:49 smithi045 ceph-mon[37795]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:49 smithi071 ceph-mon[33150]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:10:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:51 smithi071 ceph-mon[33150]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-14T15:10:51.902 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:51 smithi045 ceph-mon[37795]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-14T15:10:51.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:51 smithi045 ceph-mon[29624]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-14T15:10:51.937 INFO:teuthology.orchestra.run.smithi045.stdout:77309411365 2024-04-14T15:10:51.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.133+0000 7f61a6959700 1 -- 172.21.15.45:0/3230151056 >> v1:172.21.15.45:6790/0 conn(0x7f61a01088a0 legacy=0x7f61a0108c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.133+0000 7f61a6959700 1 -- 172.21.15.45:0/3230151056 shutdown_connections 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.133+0000 7f61a6959700 1 -- 172.21.15.45:0/3230151056 >> 172.21.15.45:0/3230151056 conn(0x7f61a00fe3d0 msgr2=0x7f61a01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.133+0000 7f61a6959700 1 -- 172.21.15.45:0/3230151056 shutdown_connections 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.133+0000 7f61a6959700 1 -- 172.21.15.45:0/3230151056 wait complete. 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f61a6959700 1 Processor -- start 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f61a6959700 1 -- start start 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f61a6959700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f61a01a9b10 con 0x7f61a010f320 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f61a6959700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f61a01aad30 con 0x7f61a01088a0 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f61a6959700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f61a01abf70 con 0x7f61a010b750 2024-04-14T15:10:52.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f619f7fe700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f61a010b750 0x7f61a01a4bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44792/0 (socket says 172.21.15.45:44792) 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.134+0000 7f619f7fe700 1 -- 172.21.15.45:0/2449614949 learned_addr learned my addr 172.21.15.45:0/2449614949 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521570706 0 0) 0x7f61a01a9b10 con 0x7f61a010f320 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f618c0035b0 con 0x7f61a010f320 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499203820 0 0) 0x7f61a01abf70 con 0x7f61a010b750 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61a01a9b10 con 0x7f61a010b750 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1279270442 0 0) 0x7f61a01aad30 con 0x7f61a01088a0 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61a01abf70 con 0x7f61a01088a0 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2465317883 0 0) 0x7f618c0035b0 con 0x7f61a010f320 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f61a01aad30 con 0x7f61a010f320 2024-04-14T15:10:52.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6198001590 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3343452388 0 0) 0x7f61a01a9b10 con 0x7f61a010b750 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f618c0035b0 con 0x7f61a010b750 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6190001830 con 0x7f61a010b750 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3190442064 0 0) 0x7f61a01aad30 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 >> v1:172.21.15.45:6790/0 conn(0x7f61a010b750 legacy=0x7f61a01a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 >> v1:172.21.15.71:6789/0 conn(0x7f61a01088a0 legacy=0x7f61a0076750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a01ad1b0 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01a9d40 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f61a01aa200 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6198003e90 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.135+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6198004da0 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.136+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f6180000fb0 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.137+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f6198003a00 con 0x7f61a010f320 2024-04-14T15:10:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.137+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f6198089190 con 0x7f61a010f320 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.137+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.71:6812/2779299537 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f618c07da30 con 0x7f618c079d80 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.137+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f6198089660 con 0x7f61a010f320 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.139+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== osd.7 v1:172.21.15.71:6812/2779299537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f618c07da30 con 0x7f618c079d80 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.155+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 --> v1:172.21.15.71:6812/2779299537 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6180002d10 con 0x7f618c079d80 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f619d7fa700 1 -- 172.21.15.45:0/2449614949 <== osd.7 v1:172.21.15.71:6812/2779299537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 2599886837) 0x7f6180002d10 con 0x7f618c079d80 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 >> v1:172.21.15.71:6812/2779299537 conn(0x7f618c079d80 legacy=0x7f618c07c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 >> v1:172.21.15.45:6800/1718216366 conn(0x7f618c06f700 legacy=0x7f618c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 >> v1:172.21.15.45:6789/0 conn(0x7f61a010f320 legacy=0x7f61a01a82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a4ef6700 1 -- 172.21.15.45:0/2449614949 reap_dead start 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 shutdown_connections 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 >> 172.21.15.45:0/2449614949 conn(0x7f61a00fe3d0 msgr2=0x7f61a0100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 shutdown_connections 2024-04-14T15:10:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:52.156+0000 7f61a6959700 1 -- 172.21.15.45:0/2449614949 wait complete. 2024-04-14T15:10:53.812 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:53 smithi045 ceph-mon[37795]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-14T15:10:53.813 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:53 smithi045 ceph-mon[29624]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-14T15:10:53.850 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:53 smithi071 ceph-mon[33150]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-14T15:10:53.851 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:10:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.241+0000 7ff963e31700 1 -- 172.21.15.45:0/590179567 >> v1:172.21.15.45:6790/0 conn(0x7ff95c103a40 legacy=0x7ff95c103e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.241+0000 7ff963e31700 1 -- 172.21.15.45:0/590179567 shutdown_connections 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.241+0000 7ff963e31700 1 -- 172.21.15.45:0/590179567 >> 172.21.15.45:0/590179567 conn(0x7ff95c0fd290 msgr2=0x7ff95c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.241+0000 7ff963e31700 1 -- 172.21.15.45:0/590179567 shutdown_connections 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.241+0000 7ff963e31700 1 -- 172.21.15.45:0/590179567 wait complete. 2024-04-14T15:10:54.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff963e31700 1 Processor -- start 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff963e31700 1 -- start start 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff963e31700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff95c1a7aa0 con 0x7ff95c103a40 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff963e31700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff95c1a8cc0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff963e31700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff95c1a9f00 con 0x7ff95c10b930 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff9613cc700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff95c10b930 0x7ff95c1a2910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44818/0 (socket says 172.21.15.45:44818) 2024-04-14T15:10:54.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff9613cc700 1 -- 172.21.15.45:0/2930777368 learned_addr learned my addr 172.21.15.45:0/2930777368 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217336080 0 0) 0x7ff95c1a9f00 con 0x7ff95c10b930 2024-04-14T15:10:54.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9480035b0 con 0x7ff95c10b930 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.242+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044676115 0 0) 0x7ff95c1a7aa0 con 0x7ff95c103a40 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95c1a9f00 con 0x7ff95c103a40 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774206690 0 0) 0x7ff95c1a8cc0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95c1a7aa0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4037092739 0 0) 0x7ff9480035b0 con 0x7ff95c10b930 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff95c1a8cc0 con 0x7ff95c10b930 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff950001930 con 0x7ff95c10b930 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 630315979 0 0) 0x7ff95c1a7aa0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff9480035b0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff954001aa0 con 0x7ff95c10f4d0 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1887661308 0 0) 0x7ff95c1a9f00 con 0x7ff95c103a40 2024-04-14T15:10:54.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff95c1a7aa0 con 0x7ff95c103a40 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff9580044e0 con 0x7ff95c103a40 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2721351639 0 0) 0x7ff95c1a8cc0 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 >> v1:172.21.15.71:6789/0 conn(0x7ff95c10f4d0 legacy=0x7ff95c1a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 >> v1:172.21.15.45:6789/0 conn(0x7ff95c103a40 legacy=0x7ff95c10b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c1ab140 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c1a7cd0 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff95c1a8190 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7ff950003b30 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.243+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff950005cc0 con 0x7ff95c10b930 2024-04-14T15:10:54.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.244+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.45:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7ff940000f80 con 0x7ff95c10b930 2024-04-14T15:10:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.245+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7ff9500036a0 con 0x7ff95c10b930 2024-04-14T15:10:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.246+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7ff95008a2a0 con 0x7ff95c10b930 2024-04-14T15:10:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.246+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.71:6808/1175230288 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff958001080 con 0x7ff94807a100 2024-04-14T15:10:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.246+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7ff95008a6e0 con 0x7ff95c10b930 2024-04-14T15:10:54.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.247+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== osd.6 v1:172.21.15.71:6808/1175230288 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7ff958001080 con 0x7ff94807a100 2024-04-14T15:10:54.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.263+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 --> v1:172.21.15.71:6808/1175230288 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff940002ce0 con 0x7ff94807a100 2024-04-14T15:10:54.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff94effd700 1 -- 172.21.15.45:0/2930777368 <== osd.6 v1:172.21.15.71:6808/1175230288 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3659474624) 0x7ff940002ce0 con 0x7ff94807a100 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 >> v1:172.21.15.71:6808/1175230288 conn(0x7ff94807a100 legacy=0x7ff94807c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 >> v1:172.21.15.45:6800/1718216366 conn(0x7ff94806fa00 legacy=0x7ff948071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 >> v1:172.21.15.45:6790/0 conn(0x7ff95c10b930 legacy=0x7ff95c1a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff9623ce700 1 -- 172.21.15.45:0/2930777368 reap_dead start 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 shutdown_connections 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 >> 172.21.15.45:0/2930777368 conn(0x7ff95c0fd290 msgr2=0x7ff95c111950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 shutdown_connections 2024-04-14T15:10:54.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.264+0000 7ff963e31700 1 -- 172.21.15.45:0/2930777368 wait complete. 2024-04-14T15:10:54.657 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:54.658 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.854+0000 7f42bd443700 1 -- 172.21.15.45:0/2452517126 >> v1:172.21.15.45:6790/0 conn(0x7f42b810f330 legacy=0x7f42b81117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.854+0000 7f42bd443700 1 -- 172.21.15.45:0/2452517126 shutdown_connections 2024-04-14T15:10:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.854+0000 7f42bd443700 1 -- 172.21.15.45:0/2452517126 >> 172.21.15.45:0/2452517126 conn(0x7f42b80fe3c0 msgr2=0x7f42b81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.854+0000 7f42bd443700 1 -- 172.21.15.45:0/2452517126 shutdown_connections 2024-04-14T15:10:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.854+0000 7f42bd443700 1 -- 172.21.15.45:0/2452517126 wait complete. 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.855+0000 7f42bd443700 1 Processor -- start 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.855+0000 7f42bd443700 1 -- start start 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.855+0000 7f42bd443700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f42b81a9b80 con 0x7f42b81088b0 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.855+0000 7f42bd443700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f42b81aada0 con 0x7f42b810f330 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.855+0000 7f42bd443700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f42b81abfe0 con 0x7f42b810b760 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42b67fc700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f42b810b760 0x7f42b81a4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44840/0 (socket says 172.21.15.45:44840) 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42b67fc700 1 -- 172.21.15.45:0/1816743406 learned_addr learned my addr 172.21.15.45:0/1816743406 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42b6ffd700 1 --1- 172.21.15.45:0/1816743406 >> v1:172.21.15.45:6789/0 conn(0x7f42b81088b0 0x7f42b810eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54898/0 (socket says 172.21.15.45:54898) 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344166605 0 0) 0x7f42b81a9b80 con 0x7f42b81088b0 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42a40035b0 con 0x7f42b81088b0 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097765801 0 0) 0x7f42b81abfe0 con 0x7f42b810b760 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b81a9b80 con 0x7f42b810b760 2024-04-14T15:10:54.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291501326 0 0) 0x7f42b81aada0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b81abfe0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 944117869 0 0) 0x7f42b81abfe0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42b81aada0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1981295662 0 0) 0x7f42a40035b0 con 0x7f42b81088b0 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42b81abfe0 con 0x7f42b81088b0 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f42b00044e0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f42a8001d10 con 0x7f42b81088b0 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 575207848 0 0) 0x7f42b81a9b80 con 0x7f42b810b760 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42a40035b0 con 0x7f42b810b760 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f42a0001870 con 0x7f42b810b760 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1089439581 0 0) 0x7f42b81aada0 con 0x7f42b810f330 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 >> v1:172.21.15.45:6790/0 conn(0x7f42b810b760 legacy=0x7f42b81a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 >> v1:172.21.15.45:6789/0 conn(0x7f42b81088b0 legacy=0x7f42b810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b81ad220 con 0x7f42b810f330 2024-04-14T15:10:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f42b81aafd0 con 0x7f42b810f330 2024-04-14T15:10:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.856+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f42b81ab490 con 0x7f42b810f330 2024-04-14T15:10:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.857+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f42b0001b70 con 0x7f42b810f330 2024-04-14T15:10:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.857+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f42b0004c70 con 0x7f42b810f330 2024-04-14T15:10:54.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.857+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.71:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f4298000fb0 con 0x7f42b810f330 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.858+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f42b0003700 con 0x7f42b810f330 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.858+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f42b0089080 con 0x7f42b810f330 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.859+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6805/1698650141 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42b0001080 con 0x7f42a407a1b0 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.859+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f42b0089460 con 0x7f42b810f330 2024-04-14T15:10:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.860+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== osd.1 v1:172.21.15.45:6805/1698650141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f42b0001080 con 0x7f42a407a1b0 2024-04-14T15:10:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.876+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 --> v1:172.21.15.45:6805/1698650141 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4298002d10 con 0x7f42a407a1b0 2024-04-14T15:10:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.876+0000 7f42affff700 1 -- 172.21.15.45:0/1816743406 <== osd.1 v1:172.21.15.45:6805/1698650141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3910016750) 0x7f4298002d10 con 0x7f42a407a1b0 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 >> v1:172.21.15.45:6805/1698650141 conn(0x7f42a407a1b0 legacy=0x7f42a407c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 >> v1:172.21.15.45:6800/1718216366 conn(0x7f42a406fab0 legacy=0x7f42a4071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 >> v1:172.21.15.71:6789/0 conn(0x7f42b810f330 legacy=0x7f42b81a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42b77fe700 1 -- 172.21.15.45:0/1816743406 reap_dead start 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 shutdown_connections 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 >> 172.21.15.45:0/1816743406 conn(0x7f42b80fe3c0 msgr2=0x7f42b8100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 shutdown_connections 2024-04-14T15:10:54.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:54.877+0000 7f42bd443700 1 -- 172.21.15.45:0/1816743406 wait complete. 2024-04-14T15:10:55.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.055+0000 7fadd085f700 1 -- 172.21.15.45:0/651316678 >> v1:172.21.15.45:6790/0 conn(0x7fadcc10a630 legacy=0x7fadcc10ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.055+0000 7fadd085f700 1 -- 172.21.15.45:0/651316678 shutdown_connections 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.055+0000 7fadd085f700 1 -- 172.21.15.45:0/651316678 >> 172.21.15.45:0/651316678 conn(0x7fadcc0fd2b0 msgr2=0x7fadcc0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.055+0000 7fadd085f700 1 -- 172.21.15.45:0/651316678 shutdown_connections 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.055+0000 7fadd085f700 1 -- 172.21.15.45:0/651316678 wait complete. 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadd085f700 1 Processor -- start 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadd085f700 1 -- start start 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadd085f700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fadcc1a9a60 con 0x7fadcc10e200 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadd085f700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fadcc1aac80 con 0x7fadcc107780 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadd085f700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fadcc1abec0 con 0x7fadcc10a630 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadc9d9b700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fadcc10a630 0x7fadcc1a4ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44850/0 (socket says 172.21.15.45:44850) 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadcad9d700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fadcc10e200 0x7fadcc1a81e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54906/0 (socket says 172.21.15.45:54906) 2024-04-14T15:10:55.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.056+0000 7fadc9d9b700 1 -- 172.21.15.45:0/1664590790 learned_addr learned my addr 172.21.15.45:0/1664590790 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028040753 0 0) 0x7fadcc1a9a60 con 0x7fadcc10e200 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb80035b0 con 0x7fadcc10e200 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2725841833 0 0) 0x7fadcc1abec0 con 0x7fadcc10a630 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadcc1a9a60 con 0x7fadcc10a630 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774578571 0 0) 0x7fadcc1aac80 con 0x7fadcc107780 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadcc1abec0 con 0x7fadcc107780 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1191809339 0 0) 0x7fadcc1a9a60 con 0x7fadcc10a630 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadcc1aac80 con 0x7fadcc10a630 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fadb40045c0 con 0x7fadcc10a630 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 199881092 0 0) 0x7fadb80035b0 con 0x7fadcc10e200 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadcc1a9a60 con 0x7fadcc10e200 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fadc4001bb0 con 0x7fadcc10e200 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3622910471 0 0) 0x7fadcc1abec0 con 0x7fadcc107780 2024-04-14T15:10:55.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadb80035b0 con 0x7fadcc107780 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fadbc0017f0 con 0x7fadcc107780 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1418636923 0 0) 0x7fadcc1aac80 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 >> v1:172.21.15.71:6789/0 conn(0x7fadcc107780 legacy=0x7fadcc10d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 >> v1:172.21.15.45:6789/0 conn(0x7fadcc10e200 legacy=0x7fadcc1a81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadcc1ad100 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fadcc1ac0f0 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fadcc1ac5b0 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fadb4001170 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.057+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fadb4004dd0 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.058+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fadac000fb0 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.059+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fadb4003680 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.059+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7fadb408a040 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.059+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6801/609845118 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadb4003b90 con 0x7fadb807a040 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.059+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7fadb408a4e0 con 0x7fadcc10a630 2024-04-14T15:10:55.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.060+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== osd.0 v1:172.21.15.45:6801/609845118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fadb4003b90 con 0x7fadb807a040 2024-04-14T15:10:55.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 --> v1:172.21.15.45:6801/609845118 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadac002d10 con 0x7fadb807a040 2024-04-14T15:10:55.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadc37fe700 1 -- 172.21.15.45:0/1664590790 <== osd.0 v1:172.21.15.45:6801/609845118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 1705684704) 0x7fadac002d10 con 0x7fadb807a040 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 >> v1:172.21.15.45:6801/609845118 conn(0x7fadb807a040 legacy=0x7fadb807c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 >> v1:172.21.15.45:6800/1718216366 conn(0x7fadb806f930 legacy=0x7fadb8071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 >> v1:172.21.15.45:6790/0 conn(0x7fadcc10a630 legacy=0x7fadcc1a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadcad9d700 1 -- 172.21.15.45:0/1664590790 reap_dead start 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.077+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 shutdown_connections 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.078+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 >> 172.21.15.45:0/1664590790 conn(0x7fadcc0fd2b0 msgr2=0x7fadcc110e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.078+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 shutdown_connections 2024-04-14T15:10:55.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.078+0000 7fadd085f700 1 -- 172.21.15.45:0/1664590790 wait complete. 2024-04-14T15:10:55.710 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:55 smithi045 ceph-mon[37795]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-14T15:10:55.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:55 smithi045 ceph-mon[29624]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-14T15:10:55.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.709+0000 7f8f212dd700 1 -- 172.21.15.45:0/3288369723 >> v1:172.21.15.45:6789/0 conn(0x7f8f1c10e1e0 legacy=0x7f8f1c1106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.709+0000 7f8f212dd700 1 -- 172.21.15.45:0/3288369723 shutdown_connections 2024-04-14T15:10:55.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.709+0000 7f8f212dd700 1 -- 172.21.15.45:0/3288369723 >> 172.21.15.45:0/3288369723 conn(0x7f8f1c0fd290 msgr2=0x7f8f1c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.709+0000 7f8f212dd700 1 -- 172.21.15.45:0/3288369723 shutdown_connections 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.709+0000 7f8f212dd700 1 -- 172.21.15.45:0/3288369723 wait complete. 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.710+0000 7f8f212dd700 1 Processor -- start 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.710+0000 7f8f212dd700 1 -- start start 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f212dd700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8f1c1a9c00 con 0x7f8f1c107760 2024-04-14T15:10:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f212dd700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8f1c1aae20 con 0x7f8f1c10a610 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f212dd700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f8f1c1ac060 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f1b7fe700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f8f1c10e1e0 0x7f8f1c1a8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44866/0 (socket says 172.21.15.45:44866) 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f1b7fe700 1 -- 172.21.15.45:0/393331983 learned_addr learned my addr 172.21.15.45:0/393331983 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296623777 0 0) 0x7f8f1c1a9c00 con 0x7f8f1c107760 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f040035b0 con 0x7f8f1c107760 2024-04-14T15:10:55.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484584345 0 0) 0x7f8f1c1ac060 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f1c1a9c00 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423738480 0 0) 0x7f8f1c1aae20 con 0x7f8f1c10a610 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f1c1ac060 con 0x7f8f1c10a610 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2929155499 0 0) 0x7f8f040035b0 con 0x7f8f1c107760 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8f1c1aae20 con 0x7f8f1c107760 2024-04-14T15:10:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f8f0c0017b0 con 0x7f8f1c107760 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3932219225 0 0) 0x7f8f1c1a9c00 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8f040035b0 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1208969093 0 0) 0x7f8f1c1ac060 con 0x7f8f1c10a610 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8f1c1a9c00 con 0x7f8f1c10a610 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f8f14004760 con 0x7f8f1c10e1e0 2024-04-14T15:10:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f8f100017c0 con 0x7f8f1c10a610 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1958529455 0 0) 0x7f8f1c1aae20 con 0x7f8f1c107760 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.711+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 >> v1:172.21.15.45:6790/0 conn(0x7f8f1c10e1e0 legacy=0x7f8f1c1a8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 >> v1:172.21.15.71:6789/0 conn(0x7f8f1c10a610 legacy=0x7f8f1c1a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c1ad2a0 con 0x7f8f1c107760 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1a9e30 con 0x7f8f1c107760 2024-04-14T15:10:55.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1aa2f0 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f8f0c003b10 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f8f0c004a50 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.712+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f8f08000f80 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.713+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f8f0c003680 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.714+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f8f0c0890f0 con 0x7f8f1c107760 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.714+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.71:6800/2605280975 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f14001400 con 0x7f8f0407a2f0 2024-04-14T15:10:55.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.714+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f8f0c089560 con 0x7f8f1c107760 2024-04-14T15:10:55.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.715+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== osd.4 v1:172.21.15.71:6800/2605280975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f8f14001400 con 0x7f8f0407a2f0 2024-04-14T15:10:55.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.733+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 --> v1:172.21.15.71:6800/2605280975 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f08002ce0 con 0x7f8f0407a2f0 2024-04-14T15:10:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f03fff700 1 -- 172.21.15.45:0/393331983 <== osd.4 v1:172.21.15.71:6800/2605280975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 1496663607) 0x7f8f08002ce0 con 0x7f8f0407a2f0 2024-04-14T15:10:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 >> v1:172.21.15.71:6800/2605280975 conn(0x7f8f0407a2f0 legacy=0x7f8f0407c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 >> v1:172.21.15.45:6800/1718216366 conn(0x7f8f0406faf0 legacy=0x7f8f04071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 >> v1:172.21.15.45:6789/0 conn(0x7f8f1c107760 legacy=0x7f8f1c10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f1b7fe700 1 -- 172.21.15.45:0/393331983 reap_dead start 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 shutdown_connections 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 >> 172.21.15.45:0/393331983 conn(0x7f8f1c0fd290 msgr2=0x7f8f1c101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:55.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 shutdown_connections 2024-04-14T15:10:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:55.734+0000 7f8f212dd700 1 -- 172.21.15.45:0/393331983 wait complete. 2024-04-14T15:10:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:55 smithi071 ceph-mon[33150]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-14T15:10:56.224 INFO:teuthology.orchestra.run.smithi045.stdout:236223201293 2024-04-14T15:10:56.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-14T15:10:56.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.396+0000 7f860b9d4700 1 -- 172.21.15.45:0/76414673 >> v1:172.21.15.45:6790/0 conn(0x7f860410a740 legacy=0x7f860410cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.396+0000 7f860b9d4700 1 -- 172.21.15.45:0/76414673 shutdown_connections 2024-04-14T15:10:56.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.396+0000 7f860b9d4700 1 -- 172.21.15.45:0/76414673 >> 172.21.15.45:0/76414673 conn(0x7f86040fd290 msgr2=0x7f86040ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:56.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.396+0000 7f860b9d4700 1 -- 172.21.15.45:0/76414673 shutdown_connections 2024-04-14T15:10:56.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.397+0000 7f860b9d4700 1 -- 172.21.15.45:0/76414673 wait complete. 2024-04-14T15:10:56.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.397+0000 7f860b9d4700 1 Processor -- start 2024-04-14T15:10:56.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.397+0000 7f860b9d4700 1 -- start start 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f860b9d4700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86041a7aa0 con 0x7f860410e310 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f860b9d4700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86041a8cc0 con 0x7f8604107890 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f860b9d4700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f86041a9f00 con 0x7f860410a740 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f8608f6f700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f860410a740 0x7f86041a27d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44898/0 (socket says 172.21.15.45:44898) 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f8609f71700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f860410e310 0x7f86041a6160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54952/0 (socket says 172.21.15.45:54952) 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f8608f6f700 1 -- 172.21.15.45:0/2804218140 learned_addr learned my addr 172.21.15.45:0/2804218140 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658943602 0 0) 0x7f86041a9f00 con 0x7f860410a740 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85f00031f0 con 0x7f860410a740 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003882176 0 0) 0x7f86041a8cc0 con 0x7f8604107890 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86041a9f00 con 0x7f8604107890 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231231846 0 0) 0x7f86041a7aa0 con 0x7f860410e310 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86041a8cc0 con 0x7f860410e310 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3118815894 0 0) 0x7f85f00031f0 con 0x7f860410a740 2024-04-14T15:10:56.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86041a7aa0 con 0x7f860410a740 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 76804195 0 0) 0x7f86041a9f00 con 0x7f8604107890 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f85f00031f0 con 0x7f8604107890 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.398+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85f40042a0 con 0x7f860410a740 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 923179706 0 0) 0x7f86041a8cc0 con 0x7f860410e310 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86041a9f00 con 0x7f860410e310 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f8600001890 con 0x7f8604107890 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2594079896 0 0) 0x7f86041a7aa0 con 0x7f860410a740 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 >> v1:172.21.15.71:6789/0 conn(0x7f8604107890 legacy=0x7f860410db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 >> v1:172.21.15.45:6789/0 conn(0x7f860410e310 legacy=0x7f86041a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86041ab140 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f86041a8ef0 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f86041a93b0 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f85f4004540 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.399+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f85f4004cb0 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.400+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.45:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f85e8000fb0 con 0x7f860410a740 2024-04-14T15:10:56.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.400+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f85f4004cb0 con 0x7f860410a740 2024-04-14T15:10:56.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.401+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f85f408a070 con 0x7f860410a740 2024-04-14T15:10:56.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.401+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.71:6804/3293828982 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85f007de60 con 0x7f85f007a1b0 2024-04-14T15:10:56.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.401+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7f85f408a4e0 con 0x7f860410a740 2024-04-14T15:10:56.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.403+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== osd.5 v1:172.21.15.71:6804/3293828982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f85f007de60 con 0x7f85f007a1b0 2024-04-14T15:10:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.421+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 --> v1:172.21.15.71:6804/3293828982 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85e8002d10 con 0x7f85f007a1b0 2024-04-14T15:10:56.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.422+0000 7f85fa7fc700 1 -- 172.21.15.45:0/2804218140 <== osd.5 v1:172.21.15.71:6804/3293828982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 3853432738) 0x7f85e8002d10 con 0x7f85f007a1b0 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.422+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 >> v1:172.21.15.71:6804/3293828982 conn(0x7f85f007a1b0 legacy=0x7f85f007c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.422+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 >> v1:172.21.15.45:6800/1718216366 conn(0x7f85f006f9b0 legacy=0x7f85f0071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 >> v1:172.21.15.45:6790/0 conn(0x7f860410a740 legacy=0x7f86041a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f8609f71700 1 -- 172.21.15.45:0/2804218140 reap_dead start 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 shutdown_connections 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 >> 172.21.15.45:0/2804218140 conn(0x7f86040fd290 msgr2=0x7f8604110f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 shutdown_connections 2024-04-14T15:10:56.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:56.423+0000 7f860b9d4700 1 -- 172.21.15.45:0/2804218140 wait complete. 2024-04-14T15:10:56.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:10:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:10:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:10:57.695 INFO:teuthology.orchestra.run.smithi045.stdout:55834574891 2024-04-14T15:10:57.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-14T15:10:57.852 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:57 smithi045 ceph-mon[37795]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:10:57.852 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:57 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:57 smithi045 ceph-mon[29624]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:10:57.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:57 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:57.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:57 smithi071 ceph-mon[33150]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:10:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:57 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.885+0000 7ff03bb10700 1 -- 172.21.15.45:0/776719763 >> v1:172.21.15.45:6789/0 conn(0x7ff03410f2f0 legacy=0x7ff0341117b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.885+0000 7ff03bb10700 1 -- 172.21.15.45:0/776719763 shutdown_connections 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.885+0000 7ff03bb10700 1 -- 172.21.15.45:0/776719763 >> 172.21.15.45:0/776719763 conn(0x7ff0340fe3a0 msgr2=0x7ff0341007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.885+0000 7ff03bb10700 1 -- 172.21.15.45:0/776719763 shutdown_connections 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 -- 172.21.15.45:0/776719763 wait complete. 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 Processor -- start 2024-04-14T15:10:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 -- start start 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0341a9ca0 con 0x7ff034108870 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0341aaec0 con 0x7ff03410b720 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03bb10700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff0341ac100 con 0x7ff03410f2f0 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff0398ac700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff034108870 0x7ff03410eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:54960/0 (socket says 172.21.15.45:54960) 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff0398ac700 1 -- 172.21.15.45:0/426040007 learned_addr learned my addr 172.21.15.45:0/426040007 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:10:57.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.886+0000 7ff03a0ad700 1 --1- 172.21.15.45:0/426040007 >> v1:172.21.15.45:6790/0 conn(0x7ff03410f2f0 0x7ff0341a8360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:44926/0 (socket says 172.21.15.45:44926) 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926842941 0 0) 0x7ff0341a9ca0 con 0x7ff034108870 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0200031f0 con 0x7ff034108870 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982112213 0 0) 0x7ff0341ac100 con 0x7ff03410f2f0 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0341a9ca0 con 0x7ff03410f2f0 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165824425 0 0) 0x7ff0341aaec0 con 0x7ff03410b720 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0341ac100 con 0x7ff03410b720 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2038273172 0 0) 0x7ff0200031f0 con 0x7ff034108870 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0341aaec0 con 0x7ff034108870 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff030001930 con 0x7ff034108870 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1714456437 0 0) 0x7ff0341a9ca0 con 0x7ff03410f2f0 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0200031f0 con 0x7ff03410f2f0 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 246688424 0 0) 0x7ff0341ac100 con 0x7ff03410b720 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0341a9ca0 con 0x7ff03410b720 2024-04-14T15:10:57.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff02c0047b0 con 0x7ff03410f2f0 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff024001890 con 0x7ff03410b720 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4003365182 0 0) 0x7ff0341aaec0 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 >> v1:172.21.15.45:6790/0 conn(0x7ff03410f2f0 legacy=0x7ff0341a8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 >> v1:172.21.15.71:6789/0 conn(0x7ff03410b720 legacy=0x7ff0341a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0341ad340 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff0341ac330 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.887+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff0341ac7f0 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.888+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7ff030003b30 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.888+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff030005c80 con 0x7ff034108870 2024-04-14T15:10:57.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.888+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7ff018000fb0 con 0x7ff034108870 2024-04-14T15:10:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.889+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7ff03001c910 con 0x7ff034108870 2024-04-14T15:10:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.889+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7ff03008a3e0 con 0x7ff034108870 2024-04-14T15:10:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.890+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6813/2647051701 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff02007e020 con 0x7ff02007a370 2024-04-14T15:10:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.890+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (unknown 2350636211 0 0) 0x7ff03008a7f0 con 0x7ff034108870 2024-04-14T15:10:57.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.891+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== osd.3 v1:172.21.15.45:6813/2647051701 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7ff02007e020 con 0x7ff02007a370 2024-04-14T15:10:57.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 --> v1:172.21.15.45:6813/2647051701 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff018002d10 con 0x7ff02007a370 2024-04-14T15:10:57.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff02affd700 1 -- 172.21.15.45:0/426040007 <== osd.3 v1:172.21.15.45:6813/2647051701 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (unknown 0 0 1221308824) 0x7ff018002d10 con 0x7ff02007a370 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 >> v1:172.21.15.45:6813/2647051701 conn(0x7ff02007a370 legacy=0x7ff02007c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 >> v1:172.21.15.45:6800/1718216366 conn(0x7ff02006fb70 legacy=0x7ff020072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 >> v1:172.21.15.45:6789/0 conn(0x7ff034108870 legacy=0x7ff03410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.907+0000 7ff03a0ad700 1 -- 172.21.15.45:0/426040007 reap_dead start 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.908+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 shutdown_connections 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.908+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 >> 172.21.15.45:0/426040007 conn(0x7ff0340fe3a0 msgr2=0x7ff034102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.908+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 shutdown_connections 2024-04-14T15:10:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:10:57.908+0000 7ff03bb10700 1 -- 172.21.15.45:0/426040007 wait complete. 2024-04-14T15:10:58.659 INFO:teuthology.orchestra.run.smithi045.stdout:133143986204 2024-04-14T15:10:58.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-14T15:10:59.059 INFO:teuthology.orchestra.run.smithi045.stdout:201863462933 2024-04-14T15:10:59.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-14T15:10:59.709 INFO:teuthology.orchestra.run.smithi045.stdout:34359738417 2024-04-14T15:10:59.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-14T15:10:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:10:59 smithi071 ceph-mon[33150]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:10:59.941 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:10:59 smithi045 ceph-mon[37795]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:10:59.941 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:10:59 smithi045 ceph-mon[29624]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:11:00.077 INFO:teuthology.orchestra.run.smithi045.stdout:167503724569 2024-04-14T15:11:00.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-14T15:11:00.458 INFO:teuthology.orchestra.run.smithi045.stdout:111669149730 2024-04-14T15:11:00.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-14T15:11:00.461 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:00.550 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:00.785 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:01.831 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:01.832 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:01 smithi071 ceph-mon[33150]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:11:01.910 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:01 smithi045 ceph-mon[37795]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:11:01.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:01 smithi045 ceph-mon[29624]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-14T15:11:02.010 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:03.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:03.639 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:04.068 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[29624]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:04.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.069 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.069 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[37795]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:04.069 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.069 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:04 smithi071 ceph-mon[33150]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.839+0000 7f45ecf49700 1 -- 172.21.15.45:0/3959777818 >> v1:172.21.15.45:6790/0 conn(0x7f45e8108880 legacy=0x7f45e8108c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.839+0000 7f45ecf49700 1 -- 172.21.15.45:0/3959777818 shutdown_connections 2024-04-14T15:11:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.839+0000 7f45ecf49700 1 -- 172.21.15.45:0/3959777818 >> 172.21.15.45:0/3959777818 conn(0x7f45e80fe3d0 msgr2=0x7f45e81007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.840+0000 7f45ecf49700 1 -- 172.21.15.45:0/3959777818 shutdown_connections 2024-04-14T15:11:04.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.840+0000 7f45ecf49700 1 -- 172.21.15.45:0/3959777818 wait complete. 2024-04-14T15:11:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45ecf49700 1 Processor -- start 2024-04-14T15:11:04.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45ecf49700 1 -- start start 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45ecf49700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f45e81a9d20 con 0x7f45e8108880 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45ecf49700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f45e81aaf40 con 0x7f45e810f300 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45ecf49700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f45e81ac180 con 0x7f45e810b730 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45e659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45e8108880 0x7f45e810eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35310/0 (socket says 172.21.15.45:35310) 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45e659c700 1 -- 172.21.15.45:0/1506241577 learned_addr learned my addr 172.21.15.45:0/1506241577 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216776838 0 0) 0x7f45e81ac180 con 0x7f45e810b730 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d00035b0 con 0x7f45e810b730 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972372785 0 0) 0x7f45e81a9d20 con 0x7f45e8108880 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45e81ac180 con 0x7f45e8108880 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370752548 0 0) 0x7f45e81aaf40 con 0x7f45e810f300 2024-04-14T15:11:04.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.841+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45e81a9d20 con 0x7f45e810f300 2024-04-14T15:11:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1506688495 0 0) 0x7f45d00035b0 con 0x7f45e810b730 2024-04-14T15:11:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f45e81aaf40 con 0x7f45e810b730 2024-04-14T15:11:04.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2161109672 0 0) 0x7f45e81ac180 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f45d00035b0 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f45dc0018d0 con 0x7f45e810b730 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2538173828 0 0) 0x7f45e81a9d20 con 0x7f45e810f300 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f45e81ac180 con 0x7f45e810f300 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f45d80044c0 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f45e0001af0 con 0x7f45e810f300 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2174148000 0 0) 0x7f45d00035b0 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 >> v1:172.21.15.45:6790/0 conn(0x7f45e810b730 legacy=0x7f45e81a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 >> v1:172.21.15.71:6789/0 conn(0x7f45e810f300 legacy=0x7f45e81a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e81ad3c0 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f45e81a9f50 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f45e81aa410 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f45d8004720 con 0x7f45e8108880 2024-04-14T15:11:04.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.842+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f45d8004ce0 con 0x7f45e8108880 2024-04-14T15:11:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.843+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8005120 con 0x7f45e8108880 2024-04-14T15:11:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.844+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f45d8004cc0 con 0x7f45e8108880 2024-04-14T15:11:04.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.844+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f45d808a200 con 0x7f45e8108880 2024-04-14T15:11:04.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:04.849+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f45d8057440 con 0x7f45e8108880 2024-04-14T15:11:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.069+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f45c8005410 con 0x7f45e8108880 2024-04-14T15:11:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.070+0000 7f45d77fe700 1 -- 172.21.15.45:0/1506241577 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 3693870858) 0x7f45d805b0f0 con 0x7f45e8108880 2024-04-14T15:11:05.071 INFO:teuthology.orchestra.run.smithi045.stdout:77309411369 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.072+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 >> v1:172.21.15.45:6800/1718216366 conn(0x7f45d006fa40 legacy=0x7f45d0071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.072+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 >> v1:172.21.15.45:6789/0 conn(0x7f45e8108880 legacy=0x7f45e810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.072+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 shutdown_connections 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.072+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 >> 172.21.15.45:0/1506241577 conn(0x7f45e80fe3d0 msgr2=0x7f45e81120c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.072+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 shutdown_connections 2024-04-14T15:11:05.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:05.073+0000 7f45ecf49700 1 -- 172.21.15.45:0/1506241577 wait complete. 2024-04-14T15:11:05.280 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:05.280 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1506241577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:06 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-04-14T15:11:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1506241577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-14T15:11:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1506241577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:06.946 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:11:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:11:07.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 3761M 2024-04-14T15:11:07.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:11:07.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 3761M 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:11:07.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:07 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 3761M 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.conf 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:07 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:07.889 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:08.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[29624]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[29624]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[29624]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[29624]: Deploying daemon alertmanager.a on smithi045 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[37795]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[37795]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[37795]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:08 smithi045 ceph-mon[37795]: Deploying daemon alertmanager.a on smithi045 2024-04-14T15:11:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:08 smithi071 ceph-mon[33150]: Updating smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:08 smithi071 ceph-mon[33150]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:08 smithi071 ceph-mon[33150]: Updating smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/config/ceph.client.admin.keyring 2024-04-14T15:11:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:08 smithi071 ceph-mon[33150]: Deploying daemon alertmanager.a on smithi045 2024-04-14T15:11:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.669+0000 7f7f1e722700 1 -- 172.21.15.45:0/2175914048 >> v1:172.21.15.45:6790/0 conn(0x7f7f1810f300 legacy=0x7f7f181117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.669+0000 7f7f1e722700 1 -- 172.21.15.45:0/2175914048 shutdown_connections 2024-04-14T15:11:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.669+0000 7f7f1e722700 1 -- 172.21.15.45:0/2175914048 >> 172.21.15.45:0/2175914048 conn(0x7f7f180fe3d0 msgr2=0x7f7f181007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.669+0000 7f7f1e722700 1 -- 172.21.15.45:0/2175914048 shutdown_connections 2024-04-14T15:11:09.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.669+0000 7f7f1e722700 1 -- 172.21.15.45:0/2175914048 wait complete. 2024-04-14T15:11:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f1e722700 1 Processor -- start 2024-04-14T15:11:09.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f1e722700 1 -- start start 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f1e722700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7f1810ef40 con 0x7f7f18108880 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f1e722700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7f181a8f10 con 0x7f7f1810b730 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f1e722700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7f181aa0f0 con 0x7f7f1810f300 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f17fff700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7f18108880 0x7f7f1810be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35330/0 (socket says 172.21.15.45:35330) 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f17fff700 1 -- 172.21.15.45:0/1710531621 learned_addr learned my addr 172.21.15.45:0/1710531621 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170262911 0 0) 0x7f7f1810ef40 con 0x7f7f18108880 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f000035b0 con 0x7f7f18108880 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.670+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645168697 0 0) 0x7f7f181aa0f0 con 0x7f7f1810f300 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f1810ef40 con 0x7f7f1810f300 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821900226 0 0) 0x7f7f181a8f10 con 0x7f7f1810b730 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f181aa0f0 con 0x7f7f1810b730 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1178101344 0 0) 0x7f7f000035b0 con 0x7f7f18108880 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f181a8f10 con 0x7f7f18108880 2024-04-14T15:11:09.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7f080018b0 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2006597251 0 0) 0x7f7f1810ef40 con 0x7f7f1810f300 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f000035b0 con 0x7f7f1810f300 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 953868193 0 0) 0x7f7f181aa0f0 con 0x7f7f1810b730 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7f1810ef40 con 0x7f7f1810b730 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7f100017b0 con 0x7f7f1810f300 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7f0c0017f0 con 0x7f7f1810b730 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 81073825 0 0) 0x7f7f181a8f10 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 >> v1:172.21.15.45:6790/0 conn(0x7f7f1810f300 legacy=0x7f7f181a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 >> v1:172.21.15.71:6789/0 conn(0x7f7f1810b730 legacy=0x7f7f1810c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f181ab2d0 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7f181aa320 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7f181aa7e0 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f7f08003b10 con 0x7f7f18108880 2024-04-14T15:11:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.671+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7f08005c50 con 0x7f7f18108880 2024-04-14T15:11:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.672+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f04005150 con 0x7f7f18108880 2024-04-14T15:11:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.673+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f7f08003680 con 0x7f7f18108880 2024-04-14T15:11:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.673+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f7f0808a080 con 0x7f7f18108880 2024-04-14T15:11:09.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.678+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f7f080572c0 con 0x7f7f18108880 2024-04-14T15:11:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.853+0000 7fbad9da1700 1 -- 172.21.15.45:0/165499754 >> v1:172.21.15.45:6790/0 conn(0x7fbad410b570 legacy=0x7fbad410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.853+0000 7fbad9da1700 1 -- 172.21.15.45:0/165499754 shutdown_connections 2024-04-14T15:11:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.853+0000 7fbad9da1700 1 -- 172.21.15.45:0/165499754 >> 172.21.15.45:0/165499754 conn(0x7fbad40fe1b0 msgr2=0x7fbad41005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.853+0000 7fbad9da1700 1 -- 172.21.15.45:0/165499754 shutdown_connections 2024-04-14T15:11:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.853+0000 7fbad9da1700 1 -- 172.21.15.45:0/165499754 wait complete. 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.854+0000 7fbad9da1700 1 Processor -- start 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.854+0000 7fbad9da1700 1 -- start start 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad9da1700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbad41a77b0 con 0x7fbad410b570 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad9da1700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbad41a89f0 con 0x7fbad410f140 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad9da1700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbad41a9c30 con 0x7fbad41086c0 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad8d9f700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbad41086c0 0x7fbad410e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60704/0 (socket says 172.21.15.45:60704) 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad8d9f700 1 -- 172.21.15.45:0/3077683575 learned_addr learned my addr 172.21.15.45:0/3077683575 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2112145337 0 0) 0x7fbad41a77b0 con 0x7fbad410b570 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbac40035b0 con 0x7fbad410b570 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576882519 0 0) 0x7fbad41a89f0 con 0x7fbad410f140 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbad41a77b0 con 0x7fbad410f140 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2526033532 0 0) 0x7fbac40035b0 con 0x7fbad410b570 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbad41a89f0 con 0x7fbad410b570 2024-04-14T15:11:09.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fbac00044a0 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1798033423 0 0) 0x7fbad41a89f0 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 >> v1:172.21.15.45:6790/0 conn(0x7fbad41086c0 legacy=0x7fbad410e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 >> v1:172.21.15.71:6789/0 conn(0x7fbad410f140 legacy=0x7fbad41a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.855+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad41aae70 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.856+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41a8c20 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.856+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fbac0001b30 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.856+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fbac0004fc0 con 0x7fbad410b570 2024-04-14T15:11:09.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.856+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbad41a90e0 con 0x7fbad410b570 2024-04-14T15:11:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.857+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fbac0003860 con 0x7fbad410b570 2024-04-14T15:11:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.857+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad4103e00 con 0x7fbad410b570 2024-04-14T15:11:09.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.857+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7fbac0089270 con 0x7fbad410b570 2024-04-14T15:11:09.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.862+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fbac0056590 con 0x7fbad410b570 2024-04-14T15:11:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.906+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7f04005440 con 0x7f7f18108880 2024-04-14T15:11:09.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.906+0000 7f7f157fa700 1 -- 172.21.15.45:0/1710531621 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (unknown 2131975755 0 270668312) 0x7f7f0805af70 con 0x7f7f18108880 2024-04-14T15:11:09.908 INFO:teuthology.orchestra.run.smithi045.stdout:167503724571 2024-04-14T15:11:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.908+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 >> v1:172.21.15.45:6800/1718216366 conn(0x7f7f0006fb70 legacy=0x7f7f00072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.908+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 >> v1:172.21.15.45:6789/0 conn(0x7f7f18108880 legacy=0x7f7f1810be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.908+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 shutdown_connections 2024-04-14T15:11:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.908+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 >> 172.21.15.45:0/1710531621 conn(0x7f7f180fe3d0 msgr2=0x7f7f181127c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.909+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 shutdown_connections 2024-04-14T15:11:09.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:09.909+0000 7f7f1e722700 1 -- 172.21.15.45:0/1710531621 wait complete. 2024-04-14T15:11:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.084+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbad410cf20 con 0x7fbad410b570 2024-04-14T15:11:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.084+0000 7fbad1ffb700 1 -- 172.21.15.45:0/3077683575 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (unknown 1482059429 0 4040121829) 0x7fbac005a240 con 0x7fbad410b570 2024-04-14T15:11:10.085 INFO:teuthology.orchestra.run.smithi045.stdout:236223201296 2024-04-14T15:11:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 >> v1:172.21.15.45:6800/1718216366 conn(0x7fbac406f600 legacy=0x7fbac4071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 >> v1:172.21.15.45:6789/0 conn(0x7fbad410b570 legacy=0x7fbad41a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 shutdown_connections 2024-04-14T15:11:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 >> 172.21.15.45:0/3077683575 conn(0x7fbad40fe1b0 msgr2=0x7fbad41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 shutdown_connections 2024-04-14T15:11:10.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:10.086+0000 7fbad9da1700 1 -- 172.21.15.45:0/3077683575 wait complete. 2024-04-14T15:11:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:10 smithi071 ceph-mon[33150]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1710531621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-14T15:11:10.440 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:10 smithi045 ceph-mon[37795]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:10.440 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1710531621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-14T15:11:10.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:10 smithi045 ceph-mon[29624]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:10.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1710531621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-14T15:11:10.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724569 got 167503724571 for osd.5 2024-04-14T15:11:10.872 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.082+0000 7fd9dbd78700 1 -- 172.21.15.45:0/908609218 >> v1:172.21.15.45:6789/0 conn(0x7fd9d4108880 legacy=0x7fd9d4108c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.082+0000 7fd9dbd78700 1 -- 172.21.15.45:0/908609218 shutdown_connections 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.082+0000 7fd9dbd78700 1 -- 172.21.15.45:0/908609218 >> 172.21.15.45:0/908609218 conn(0x7fd9d40fe3d0 msgr2=0x7fd9d41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.082+0000 7fd9dbd78700 1 -- 172.21.15.45:0/908609218 shutdown_connections 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.082+0000 7fd9dbd78700 1 -- 172.21.15.45:0/908609218 wait complete. 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9dbd78700 1 Processor -- start 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9dbd78700 1 -- start start 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9dbd78700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9d41a7990 con 0x7fd9d4108880 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9dbd78700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9d41a8bb0 con 0x7fd9d410b730 2024-04-14T15:11:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9dbd78700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd9d41a9df0 con 0x7fd9d410f300 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9d9b14700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd9d4108880 0x7fd9d410eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53388/0 (socket says 172.21.15.45:53388) 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9d9b14700 1 -- 172.21.15.45:0/3397721475 learned_addr learned my addr 172.21.15.45:0/3397721475 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651806241 0 0) 0x7fd9d41a9df0 con 0x7fd9d410f300 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c00031f0 con 0x7fd9d410f300 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787250816 0 0) 0x7fd9d41a7990 con 0x7fd9d4108880 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d41a9df0 con 0x7fd9d4108880 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006932843 0 0) 0x7fd9d41a8bb0 con 0x7fd9d410b730 2024-04-14T15:11:11.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.083+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d41a7990 con 0x7fd9d410b730 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3984847205 0 0) 0x7fd9c00031f0 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9d41a8bb0 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd9cc001b70 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2825521114 0 0) 0x7fd9d41a9df0 con 0x7fd9d4108880 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9c00031f0 con 0x7fd9d4108880 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd9d0004520 con 0x7fd9d4108880 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3780584631 0 0) 0x7fd9d41a7990 con 0x7fd9d410b730 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd9d41a9df0 con 0x7fd9d410b730 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd9c4001230 con 0x7fd9d410b730 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 142395727 0 0) 0x7fd9d41a8bb0 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 >> v1:172.21.15.71:6789/0 conn(0x7fd9d410b730 legacy=0x7fd9d41a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 >> v1:172.21.15.45:6789/0 conn(0x7fd9d4108880 legacy=0x7fd9d410eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d41ab030 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d41a7bc0 con 0x7fd9d410f300 2024-04-14T15:11:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd9d41a8080 con 0x7fd9d410f300 2024-04-14T15:11:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd9cc003e90 con 0x7fd9d410f300 2024-04-14T15:11:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.084+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd9cc004dc0 con 0x7fd9d410f300 2024-04-14T15:11:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.085+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b8005120 con 0x7fd9d410f300 2024-04-14T15:11:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.085+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fd9cc003a00 con 0x7fd9d410f300 2024-04-14T15:11:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.086+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7fd9cc089300 con 0x7fd9d410f300 2024-04-14T15:11:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.091+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fd9cc0564e0 con 0x7fd9d410f300 2024-04-14T15:11:11.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3077683575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-14T15:11:11.195 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3077683575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-14T15:11:11.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.300+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd9b8005410 con 0x7fd9d410f300 2024-04-14T15:11:11.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.301+0000 7fd9caffd700 1 -- 172.21.15.45:0/3397721475 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 2037106864) 0x7fd9cc05a190 con 0x7fd9d410f300 2024-04-14T15:11:11.302 INFO:teuthology.orchestra.run.smithi045.stdout:55834574894 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 >> v1:172.21.15.45:6800/1718216366 conn(0x7fd9c006fae0 legacy=0x7fd9c0071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 >> v1:172.21.15.45:6790/0 conn(0x7fd9d410f300 legacy=0x7fd9d41a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 shutdown_connections 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 >> 172.21.15.45:0/3397721475 conn(0x7fd9d40fe3d0 msgr2=0x7fd9d4111f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 shutdown_connections 2024-04-14T15:11:11.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:11.303+0000 7fd9dbd78700 1 -- 172.21.15.45:0/3397721475 wait complete. 2024-04-14T15:11:11.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3077683575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-14T15:11:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:12 smithi071 ceph-mon[33150]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3397721475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-14T15:11:12.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:12 smithi045 ceph-mon[29624]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:12.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3397721475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-14T15:11:12.442 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:12 smithi045 ceph-mon[37795]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:12.442 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3397721475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-14T15:11:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.548+0000 7f6054b48700 1 -- 172.21.15.45:0/3532067691 >> v1:172.21.15.45:6790/0 conn(0x7f605010e1e0 legacy=0x7f60501106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.548+0000 7f6054b48700 1 -- 172.21.15.45:0/3532067691 shutdown_connections 2024-04-14T15:11:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.548+0000 7f6054b48700 1 -- 172.21.15.45:0/3532067691 >> 172.21.15.45:0/3532067691 conn(0x7f60500fd290 msgr2=0x7f60500ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.548+0000 7f6054b48700 1 -- 172.21.15.45:0/3532067691 shutdown_connections 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.548+0000 7f6054b48700 1 -- 172.21.15.45:0/3532067691 wait complete. 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f6054b48700 1 Processor -- start 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f6054b48700 1 -- start start 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f6054b48700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f60501a6840 con 0x7f605010e1e0 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f6054b48700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f60501a7a60 con 0x7f605010a610 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f6054b48700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f60501a8ca0 con 0x7f6050107760 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f604e59c700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6050107760 0x7f605010d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:38206/0 (socket says 172.21.15.45:38206) 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f604e59c700 1 -- 172.21.15.45:0/2352226451 learned_addr learned my addr 172.21.15.45:0/2352226451 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454604991 0 0) 0x7f60501a8ca0 con 0x7f6050107760 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60380031f0 con 0x7f6050107760 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.549+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 406458657 0 0) 0x7f60501a6840 con 0x7f605010e1e0 2024-04-14T15:11:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60501a8ca0 con 0x7f605010e1e0 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152876261 0 0) 0x7f60501a7a60 con 0x7f605010a610 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60501a6840 con 0x7f605010a610 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2851276620 0 0) 0x7f60380031f0 con 0x7f6050107760 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60501a7a60 con 0x7f6050107760 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6040001a30 con 0x7f6050107760 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 996996256 0 0) 0x7f60501a8ca0 con 0x7f605010e1e0 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60380031f0 con 0x7f605010e1e0 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6048001c00 con 0x7f605010e1e0 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1419705912 0 0) 0x7f60501a6840 con 0x7f605010a610 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f60501a8ca0 con 0x7f605010a610 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6044001830 con 0x7f605010a610 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1129565757 0 0) 0x7f60501a7a60 con 0x7f6050107760 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 >> v1:172.21.15.71:6789/0 conn(0x7f605010a610 legacy=0x7f60501a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 >> v1:172.21.15.45:6789/0 conn(0x7f605010e1e0 legacy=0x7f60501a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501a9ee0 con 0x7f6050107760 2024-04-14T15:11:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a8ed0 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f60501a9390 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6040001bd0 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.550+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6040005b40 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.551+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6030005120 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.552+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f604001c7d0 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.552+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f604008a0a0 con 0x7f6050107760 2024-04-14T15:11:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.557+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f60400572e0 con 0x7f6050107760 2024-04-14T15:11:12.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.801+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6030005410 con 0x7f6050107760 2024-04-14T15:11:12.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.802+0000 7f603f7fe700 1 -- 172.21.15.45:0/2352226451 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (unknown 1823589948 0 3145046205) 0x7f604005af90 con 0x7f6050107760 2024-04-14T15:11:12.803 INFO:teuthology.orchestra.run.smithi045.stdout:133143986207 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.804+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 >> v1:172.21.15.45:6800/1718216366 conn(0x7f6038071ba0 legacy=0x7f6038074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.804+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 >> v1:172.21.15.45:6790/0 conn(0x7f6050107760 legacy=0x7f605010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.804+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 shutdown_connections 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.804+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 >> 172.21.15.45:0/2352226451 conn(0x7f60500fd290 msgr2=0x7f6050101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.804+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 shutdown_connections 2024-04-14T15:11:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.805+0000 7f6054b48700 1 -- 172.21.15.45:0/2352226451 wait complete. 2024-04-14T15:11:12.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.907+0000 7f08241cf700 1 -- 172.21.15.45:0/631911325 >> v1:172.21.15.45:6789/0 conn(0x7f081c10f340 legacy=0x7f081c1117d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.907+0000 7f08241cf700 1 -- 172.21.15.45:0/631911325 shutdown_connections 2024-04-14T15:11:12.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.907+0000 7f08241cf700 1 -- 172.21.15.45:0/631911325 >> 172.21.15.45:0/631911325 conn(0x7f081c0fd290 msgr2=0x7f081c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.907+0000 7f08241cf700 1 -- 172.21.15.45:0/631911325 shutdown_connections 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.907+0000 7f08241cf700 1 -- 172.21.15.45:0/631911325 wait complete. 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 Processor -- start 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- start start 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f081c1a77f0 con 0x7f081c103a20 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f081c1a8a10 con 0x7f081c10b830 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f081c1a9c50 con 0x7f081c10f340 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0821f6b700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f081c103a20 0x7f081c10af00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53410/0 (socket says 172.21.15.45:53410) 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0821f6b700 1 -- 172.21.15.45:0/1748276567 learned_addr learned my addr 172.21.15.45:0/1748276567 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f082276c700 1 --1- 172.21.15.45:0/1748276567 >> v1:172.21.15.45:6790/0 conn(0x7f081c10f340 0x7f081c1a5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:38228/0 (socket says 172.21.15.45:38228) 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013558888 0 0) 0x7f081c1a9c50 con 0x7f081c10f340 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08080035b0 con 0x7f081c10f340 2024-04-14T15:11:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721275044 0 0) 0x7f081c1a8a10 con 0x7f081c10b830 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f081c1a9c50 con 0x7f081c10b830 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568624501 0 0) 0x7f081c1a77f0 con 0x7f081c103a20 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f081c1a8a10 con 0x7f081c103a20 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1957778357 0 0) 0x7f08080035b0 con 0x7f081c10f340 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f081c1a77f0 con 0x7f081c10f340 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0814004860 con 0x7f081c10f340 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3419826237 0 0) 0x7f081c1a8a10 con 0x7f081c103a20 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08080035b0 con 0x7f081c103a20 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2783684095 0 0) 0x7f081c1a9c50 con 0x7f081c10b830 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f081c1a8a10 con 0x7f081c10b830 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f08180019d0 con 0x7f081c103a20 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f080c001130 con 0x7f081c10b830 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 327944871 0 0) 0x7f081c1a77f0 con 0x7f081c10f340 2024-04-14T15:11:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 >> v1:172.21.15.71:6789/0 conn(0x7f081c10b830 legacy=0x7f081c1a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 >> v1:172.21.15.45:6789/0 conn(0x7f081c103a20 legacy=0x7f081c10af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c1aae90 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f081c1a8c40 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.909+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f081c1a9100 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.910+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0814001ef0 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.910+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f08140051c0 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.910+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081c06ad40 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.912+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f0814005360 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.913+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f081408a5b0 con 0x7f081c10f340 2024-04-14T15:11:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:12.918+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f08140577f0 con 0x7f081c10f340 2024-04-14T15:11:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.138+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f081c1094e0 con 0x7f081c10f340 2024-04-14T15:11:13.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.139+0000 7f0812ffd700 1 -- 172.21.15.45:0/1748276567 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 418653114) 0x7f081405b4a0 con 0x7f081c10f340 2024-04-14T15:11:13.139 INFO:teuthology.orchestra.run.smithi045.stdout:34359738420 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 >> v1:172.21.15.45:6800/1718216366 conn(0x7f080806fa00 legacy=0x7f0808071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 >> v1:172.21.15.45:6790/0 conn(0x7f081c10f340 legacy=0x7f081c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 shutdown_connections 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 >> 172.21.15.45:0/1748276567 conn(0x7f081c0fd290 msgr2=0x7f081c111790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 shutdown_connections 2024-04-14T15:11:13.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:13.141+0000 7f08241cf700 1 -- 172.21.15.45:0/1748276567 wait complete. 2024-04-14T15:11:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2352226451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-14T15:11:13.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:13.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2352226451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-14T15:11:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2352226451' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-14T15:11:13.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:14.066 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738420 for osd.0 2024-04-14T15:11:14.066 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:14 smithi071 ceph-mon[33150]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1748276567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-14T15:11:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:14 smithi045 ceph-mon[37795]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1748276567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-14T15:11:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:14 smithi045 ceph-mon[29624]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1748276567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-14T15:11:15.306 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:15.307 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:16 smithi071 ceph-mon[33150]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:16.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:16 smithi045 ceph-mon[29624]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:16.435 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:16 smithi045 ceph-mon[37795]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:16.468 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574894 for osd.1 2024-04-14T15:11:16.468 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:16.995 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:11:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:11:16.996 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201293 got 236223201296 for osd.7 2024-04-14T15:11:16.997 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:16.997 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986204 got 133143986207 for osd.4 2024-04-14T15:11:16.997 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:17.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411365 got 77309411369 for osd.2 2024-04-14T15:11:17.744 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:18 smithi071 ceph-mon[33150]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:18 smithi045 ceph-mon[37795]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:18 smithi045 ceph-mon[29624]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.746+0000 7f76413ea700 1 -- 172.21.15.45:0/1513392830 >> v1:172.21.15.45:6789/0 conn(0x7f763c10a610 legacy=0x7f763c10ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.746+0000 7f76413ea700 1 -- 172.21.15.45:0/1513392830 shutdown_connections 2024-04-14T15:11:18.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.746+0000 7f76413ea700 1 -- 172.21.15.45:0/1513392830 >> 172.21.15.45:0/1513392830 conn(0x7f763c0fd290 msgr2=0x7f763c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.746+0000 7f76413ea700 1 -- 172.21.15.45:0/1513392830 shutdown_connections 2024-04-14T15:11:18.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.746+0000 7f76413ea700 1 -- 172.21.15.45:0/1513392830 wait complete. 2024-04-14T15:11:18.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f76413ea700 1 Processor -- start 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f76413ea700 1 -- start start 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f76413ea700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f763c1a7930 con 0x7f763c107760 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f76413ea700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f763c1a8b70 con 0x7f763c10a610 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f76413ea700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f763c1a9db0 con 0x7f763c10e1e0 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f763affd700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f763c107760 0x7f763c10da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53428/0 (socket says 172.21.15.45:53428) 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.747+0000 7f763affd700 1 -- 172.21.15.45:0/352612151 learned_addr learned my addr 172.21.15.45:0/352612151 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1383373442 0 0) 0x7f763c1a7930 con 0x7f763c107760 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76280035b0 con 0x7f763c107760 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940916453 0 0) 0x7f763c1a9db0 con 0x7f763c10e1e0 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763c1a7930 con 0x7f763c10e1e0 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3601124765 0 0) 0x7f763c1a8b70 con 0x7f763c10a610 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763c1a9db0 con 0x7f763c10a610 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2591712723 0 0) 0x7f76280035b0 con 0x7f763c107760 2024-04-14T15:11:18.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f763c1a8b70 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f762c001cb0 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2578507676 0 0) 0x7f763c1a7930 con 0x7f763c10e1e0 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76280035b0 con 0x7f763c10e1e0 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f76340010f0 con 0x7f763c10e1e0 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 505742815 0 0) 0x7f763c1a9db0 con 0x7f763c10a610 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f763c1a7930 con 0x7f763c10a610 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7624004540 con 0x7f763c10a610 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2945262620 0 0) 0x7f763c1a8b70 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 >> v1:172.21.15.45:6790/0 conn(0x7f763c10e1e0 legacy=0x7f763c1a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 >> v1:172.21.15.71:6789/0 conn(0x7f763c10a610 legacy=0x7f763c1a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c1aaff0 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f763c1a9fe0 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f763c1aa4a0 con 0x7f763c107760 2024-04-14T15:11:18.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.748+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f762c003eb0 con 0x7f763c107760 2024-04-14T15:11:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.749+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f762c006040 con 0x7f763c107760 2024-04-14T15:11:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.749+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c005150 con 0x7f763c107760 2024-04-14T15:11:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.750+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f762c003a20 con 0x7f763c107760 2024-04-14T15:11:18.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.750+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f762c08a4d0 con 0x7f763c107760 2024-04-14T15:11:18.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.755+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f762c057710 con 0x7f763c107760 2024-04-14T15:11:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.974+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f761c005440 con 0x7f763c107760 2024-04-14T15:11:18.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.974+0000 7f7633fff700 1 -- 172.21.15.45:0/352612151 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (unknown 1274345170 0 3862917126) 0x7f762c05b3c0 con 0x7f763c107760 2024-04-14T15:11:18.974 INFO:teuthology.orchestra.run.smithi045.stdout:201863462937 2024-04-14T15:11:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 >> v1:172.21.15.45:6800/1718216366 conn(0x7f762806f9b0 legacy=0x7f7628071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 >> v1:172.21.15.45:6789/0 conn(0x7f763c107760 legacy=0x7f763c10da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 shutdown_connections 2024-04-14T15:11:18.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 >> 172.21.15.45:0/352612151 conn(0x7f763c0fd290 msgr2=0x7f763c110e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 shutdown_connections 2024-04-14T15:11:18.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:18.976+0000 7f76413ea700 1 -- 172.21.15.45:0/352612151 wait complete. 2024-04-14T15:11:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/352612151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-14T15:11:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/352612151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-14T15:11:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/352612151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-14T15:11:20.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:20 smithi045 ceph-mon[29624]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:20.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:20 smithi045 ceph-mon[37795]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:20 smithi071 ceph-mon[33150]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:20.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.367+0000 7f7ab6955700 1 -- 172.21.15.45:0/3873300514 >> v1:172.21.15.45:6790/0 conn(0x7f7ab010a790 legacy=0x7f7ab010cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:20.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 -- 172.21.15.45:0/3873300514 shutdown_connections 2024-04-14T15:11:20.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 -- 172.21.15.45:0/3873300514 >> 172.21.15.45:0/3873300514 conn(0x7f7ab00fd2c0 msgr2=0x7f7ab00ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:20.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 -- 172.21.15.45:0/3873300514 shutdown_connections 2024-04-14T15:11:20.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 -- 172.21.15.45:0/3873300514 wait complete. 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 Processor -- start 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.368+0000 7f7ab6955700 1 -- start start 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7ab6955700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab01a9b90 con 0x7f7ab010e360 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7ab6955700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab01aadb0 con 0x7f7ab010a790 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7ab6955700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f7ab01abfd0 con 0x7f7ab01078e0 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aaffff700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f7ab01078e0 0x7f7ab010d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:38252/0 (socket says 172.21.15.45:38252) 2024-04-14T15:11:20.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aaffff700 1 -- 172.21.15.45:0/1753928323 learned_addr learned my addr 172.21.15.45:0/1753928323 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492209125 0 0) 0x7f7ab01abfd0 con 0x7f7ab01078e0 2024-04-14T15:11:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a980031f0 con 0x7f7ab01078e0 2024-04-14T15:11:20.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2885241622 0 0) 0x7f7ab01a9b90 con 0x7f7ab010e360 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab01abfd0 con 0x7f7ab010e360 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731724230 0 0) 0x7f7ab01aadb0 con 0x7f7ab010a790 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab01a9b90 con 0x7f7ab010a790 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2620909765 0 0) 0x7f7a980031f0 con 0x7f7ab01078e0 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ab01aadb0 con 0x7f7ab01078e0 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7aa0001930 con 0x7f7ab01078e0 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2674169642 0 0) 0x7f7ab01a9b90 con 0x7f7ab010a790 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a980031f0 con 0x7f7ab010a790 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2592471532 0 0) 0x7f7ab01abfd0 con 0x7f7ab010e360 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ab01a9b90 con 0x7f7ab010e360 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.369+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7aa4001980 con 0x7f7ab010a790 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7aa8001470 con 0x7f7ab010e360 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 616727552 0 0) 0x7f7ab01aadb0 con 0x7f7ab01078e0 2024-04-14T15:11:20.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 >> v1:172.21.15.71:6789/0 conn(0x7f7ab010a790 legacy=0x7f7ab01a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 >> v1:172.21.15.45:6789/0 conn(0x7f7ab010e360 legacy=0x7f7ab01a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01ad210 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab01a9dc0 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7ab01aa280 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f7aa0001d00 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.370+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f7aa0005da0 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.371+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9c005120 con 0x7f7ab01078e0 2024-04-14T15:11:20.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.371+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f7aa001c9f0 con 0x7f7ab01078e0 2024-04-14T15:11:20.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.372+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f7aa008a290 con 0x7f7ab01078e0 2024-04-14T15:11:20.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.376+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f7aa00574d0 con 0x7f7ab01078e0 2024-04-14T15:11:20.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.585+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7a9c005410 con 0x7f7ab01078e0 2024-04-14T15:11:20.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.586+0000 7f7aad7fa700 1 -- 172.21.15.45:0/1753928323 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (unknown 383520633 0 1028728461) 0x7f7aa005b180 con 0x7f7ab01078e0 2024-04-14T15:11:20.586 INFO:teuthology.orchestra.run.smithi045.stdout:111669149734 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 >> v1:172.21.15.45:6800/1718216366 conn(0x7f7a9806f9b0 legacy=0x7f7a98071e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 >> v1:172.21.15.45:6790/0 conn(0x7f7ab01078e0 legacy=0x7f7ab010d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 shutdown_connections 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 >> 172.21.15.45:0/1753928323 conn(0x7f7ab00fd2c0 msgr2=0x7f7ab01017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 shutdown_connections 2024-04-14T15:11:20.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:20.588+0000 7f7ab6955700 1 -- 172.21.15.45:0/1753928323 wait complete. 2024-04-14T15:11:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1753928323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-14T15:11:21.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1753928323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-14T15:11:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1753928323' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-14T15:11:21.932 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:21 smithi045 systemd[1]: Starting Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:22.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:22 smithi045 ceph-mon[29624]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:22.199 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:22 smithi045 ceph-mon[37795]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:22.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149734 for osd.3 2024-04-14T15:11:22.238 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:22 smithi071 ceph-mon[33150]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:22.658 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462933 got 201863462937 for osd.6 2024-04-14T15:11:22.658 DEBUG:teuthology.parallel:result is None 2024-04-14T15:11:22.658 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-14T15:11:22.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph pg dump --format=json 2024-04-14T15:11:22.936 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:22 smithi045 podman[72043]: 2024-04-14 15:11:22.828105115 +0000 UTC m=+0.110346931 volume create 27cd2a3cb7c6cca96ffdf427c86eb0cdce59bfcefb99522691f32b8fadbf661a 2024-04-14T15:11:22.936 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:22 smithi045 podman[72043]: 2024-04-14 15:11:22.739314009 +0000 UTC m=+0.021555851 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-04-14T15:11:22.937 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:22 smithi045 podman[72043]: 2024-04-14 15:11:22.937082977 +0000 UTC m=+0.219324824 container create beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:23.465 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 podman[72043]: 2024-04-14 15:11:23.220652127 +0000 UTC m=+0.502893931 container init beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 podman[72043]: 2024-04-14 15:11:23.223308885 +0000 UTC m=+0.505550698 container start beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.235Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.235Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.236Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.45 port=9094 2024-04-14T15:11:23.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.237Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.261Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.261Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.264Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:23.264Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 bash[72043]: beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb 2024-04-14T15:11:23.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:23 smithi045 systemd[1]: Started Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:23.809 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE Bus STOPPING 2024-04-14T15:11:23.810 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:11:23.810 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE Bus STOPPED 2024-04-14T15:11:23.810 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE Bus STARTING 2024-04-14T15:11:23.810 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE Serving on http://:::9283 2024-04-14T15:11:23.810 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:23] ENGINE Bus STARTED 2024-04-14T15:11:23.856 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:24 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:24.465 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:24 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/2969021041 >> v1:172.21.15.45:6789/0 conn(0x7f9b0410e200 legacy=0x7f9b041106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/2969021041 shutdown_connections 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/2969021041 >> 172.21.15.45:0/2969021041 conn(0x7f9b040fd2b0 msgr2=0x7f9b040ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/2969021041 shutdown_connections 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/2969021041 wait complete. 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 Processor -- start 2024-04-14T15:11:24.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.971+0000 7f9b0a9ea700 1 -- start start 2024-04-14T15:11:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b0a9ea700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9b041a6840 con 0x7f9b0410e200 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b0a9ea700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9b041a7a60 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b0a9ea700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9b041a8ca0 con 0x7f9b04107780 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b03fff700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f9b04107780 0x7f9b0410d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:35082/0 (socket says 172.21.15.45:35082) 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b03fff700 1 -- 172.21.15.45:0/475397246 learned_addr learned my addr 172.21.15.45:0/475397246 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792798693 0 0) 0x7f9b041a7a60 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9af00035b0 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323557932 0 0) 0x7f9b041a8ca0 con 0x7f9b04107780 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b041a7a60 con 0x7f9b04107780 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687907918 0 0) 0x7f9b041a6840 con 0x7f9b0410e200 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b041a8ca0 con 0x7f9b0410e200 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 276161005 0 0) 0x7f9af00035b0 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b041a6840 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9af40018b0 con 0x7f9b0410a630 2024-04-14T15:11:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4145066394 0 0) 0x7f9b041a8ca0 con 0x7f9b0410e200 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9af00035b0 con 0x7f9b0410e200 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1026684861 0 0) 0x7f9b041a7a60 con 0x7f9b04107780 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b041a8ca0 con 0x7f9b04107780 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9afc001bc0 con 0x7f9b0410e200 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.972+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9aec001a10 con 0x7f9b04107780 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3995007609 0 0) 0x7f9b041a6840 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 >> v1:172.21.15.45:6790/0 conn(0x7f9b04107780 legacy=0x7f9b0410d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 >> v1:172.21.15.45:6789/0 conn(0x7f9b0410e200 legacy=0x7f9b041a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041a9ee0 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041a8ed0 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f9b041a9390 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9af4003b30 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9af40049c0 con 0x7f9b0410a630 2024-04-14T15:11:24.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.973+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.71:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae4005150 con 0x7f9b0410a630 2024-04-14T15:11:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.974+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f9af40036a0 con 0x7f9b0410a630 2024-04-14T15:11:24.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.975+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7f9af40890e0 con 0x7f9b0410a630 2024-04-14T15:11:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:24.979+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mon.1 v1:172.21.15.71:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9af40562c0 con 0x7f9b0410a630 2024-04-14T15:11:25.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.183+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 --> v1:172.21.15.45:6800/1718216366 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ae4000bc0 con 0x7f9af0071d50 2024-04-14T15:11:25.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.192+0000 7f9b017fa700 1 -- 172.21.15.45:0/475397246 <== mgr.14610 v1:172.21.15.45:6800/1718216366 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346474 (unknown 2965378022 0 4172647688) 0x7f9ae4000bc0 con 0x7f9af0071d50 2024-04-14T15:11:25.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 >> v1:172.21.15.45:6800/1718216366 conn(0x7f9af0071d50 legacy=0x7f9af0074210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 >> v1:172.21.15.71:6789/0 conn(0x7f9b0410a630 legacy=0x7f9b041a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 shutdown_connections 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 >> 172.21.15.45:0/475397246 conn(0x7f9b040fd2b0 msgr2=0x7f9b04110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 shutdown_connections 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:25.196+0000 7f9b0a9ea700 1 -- 172.21.15.45:0/475397246 wait complete. 2024-04-14T15:11:25.202 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-14T15:11:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:25 smithi071 ceph-mon[33150]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-14T15:11:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:25 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:25 smithi071 ceph-mon[33150]: Deploying daemon grafana.a on smithi071 2024-04-14T15:11:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[29624]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-14T15:11:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[29624]: Deploying daemon grafana.a on smithi071 2024-04-14T15:11:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[37795]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-14T15:11:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-14T15:11:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-mon[37795]: Deploying daemon grafana.a on smithi071 2024-04-14T15:11:25.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:25 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:25.238Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00034107s 2024-04-14T15:11:25.833 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-14T15:11:24.354243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1010,"num_read_kb":856,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235804,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725412,"statfs":{"total":767960285184,"available":767718821888,"internally_reserved":0,"allocated":7335936,"data_stored":3908472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002739"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505382+0000","last_change":"2024-04-14T15:10:07.328416+0000","last_active":"2024-04-14T15:10:42.505382+0000","last_peered":"2024-04-14T15:10:42.505382+0000","last_clean":"2024-04-14T15:10:42.505382+0000","last_became_active":"2024-04-14T15:10:07.328053+0000","last_became_peered":"2024-04-14T15:10:07.328053+0000","last_unstale":"2024-04-14T15:10:42.505382+0000","last_undegraded":"2024-04-14T15:10:42.505382+0000","last_fullsized":"2024-04-14T15:10:42.505382+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:35:00.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335717+0000","last_change":"2024-04-14T15:10:05.316678+0000","last_active":"2024-04-14T15:10:42.335717+0000","last_peered":"2024-04-14T15:10:42.335717+0000","last_clean":"2024-04-14T15:10:42.335717+0000","last_became_active":"2024-04-14T15:10:05.314691+0000","last_became_peered":"2024-04-14T15:10:05.314691+0000","last_unstale":"2024-04-14T15:10:42.335717+0000","last_undegraded":"2024-04-14T15:10:42.335717+0000","last_fullsized":"2024-04-14T15:10:42.335717+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:23:43.814674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803653+0000","last_change":"2024-04-14T15:10:01.290234+0000","last_active":"2024-04-14T15:10:43.803653+0000","last_peered":"2024-04-14T15:10:43.803653+0000","last_clean":"2024-04-14T15:10:43.803653+0000","last_became_active":"2024-04-14T15:10:01.289946+0000","last_became_peered":"2024-04-14T15:10:01.289946+0000","last_unstale":"2024-04-14T15:10:43.803653+0000","last_undegraded":"2024-04-14T15:10:43.803653+0000","last_fullsized":"2024-04-14T15:10:43.803653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:38:09.696997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505252+0000","last_change":"2024-04-14T15:10:03.306513+0000","last_active":"2024-04-14T15:10:42.505252+0000","last_peered":"2024-04-14T15:10:42.505252+0000","last_clean":"2024-04-14T15:10:42.505252+0000","last_became_active":"2024-04-14T15:10:03.306066+0000","last_became_peered":"2024-04-14T15:10:03.306066+0000","last_unstale":"2024-04-14T15:10:42.505252+0000","last_undegraded":"2024-04-14T15:10:42.505252+0000","last_fullsized":"2024-04-14T15:10:42.505252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:01:46.636333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804789+0000","last_change":"2024-04-14T15:10:07.639197+0000","last_active":"2024-04-14T15:10:43.804789+0000","last_peered":"2024-04-14T15:10:43.804789+0000","last_clean":"2024-04-14T15:10:43.804789+0000","last_became_active":"2024-04-14T15:10:07.638812+0000","last_became_peered":"2024-04-14T15:10:07.638812+0000","last_unstale":"2024-04-14T15:10:43.804789+0000","last_undegraded":"2024-04-14T15:10:43.804789+0000","last_fullsized":"2024-04-14T15:10:43.804789+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:14:16.938555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505273+0000","last_change":"2024-04-14T15:10:05.313030+0000","last_active":"2024-04-14T15:10:42.505273+0000","last_peered":"2024-04-14T15:10:42.505273+0000","last_clean":"2024-04-14T15:10:42.505273+0000","last_became_active":"2024-04-14T15:10:05.312720+0000","last_became_peered":"2024-04-14T15:10:05.312720+0000","last_unstale":"2024-04-14T15:10:42.505273+0000","last_undegraded":"2024-04-14T15:10:42.505273+0000","last_fullsized":"2024-04-14T15:10:42.505273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:14.331280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804841+0000","last_change":"2024-04-14T15:10:01.287434+0000","last_active":"2024-04-14T15:10:43.804841+0000","last_peered":"2024-04-14T15:10:43.804841+0000","last_clean":"2024-04-14T15:10:43.804841+0000","last_became_active":"2024-04-14T15:10:01.287134+0000","last_became_peered":"2024-04-14T15:10:01.287134+0000","last_unstale":"2024-04-14T15:10:43.804841+0000","last_undegraded":"2024-04-14T15:10:43.804841+0000","last_fullsized":"2024-04-14T15:10:43.804841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:14:59.013867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803115+0000","last_change":"2024-04-14T15:10:03.298079+0000","last_active":"2024-04-14T15:10:43.803115+0000","last_peered":"2024-04-14T15:10:43.803115+0000","last_clean":"2024-04-14T15:10:43.803115+0000","last_became_active":"2024-04-14T15:10:03.297788+0000","last_became_peered":"2024-04-14T15:10:03.297788+0000","last_unstale":"2024-04-14T15:10:43.803115+0000","last_undegraded":"2024-04-14T15:10:43.803115+0000","last_fullsized":"2024-04-14T15:10:43.803115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:24:54.677004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804940+0000","last_change":"2024-04-14T15:10:07.327110+0000","last_active":"2024-04-14T15:10:43.804940+0000","last_peered":"2024-04-14T15:10:43.804940+0000","last_clean":"2024-04-14T15:10:43.804940+0000","last_became_active":"2024-04-14T15:10:07.326743+0000","last_became_peered":"2024-04-14T15:10:07.326743+0000","last_unstale":"2024-04-14T15:10:43.804940+0000","last_undegraded":"2024-04-14T15:10:43.804940+0000","last_fullsized":"2024-04-14T15:10:43.804940+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:27:40.355052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426656+0000","last_change":"2024-04-14T15:10:05.314296+0000","last_active":"2024-04-14T15:10:42.426656+0000","last_peered":"2024-04-14T15:10:42.426656+0000","last_clean":"2024-04-14T15:10:42.426656+0000","last_became_active":"2024-04-14T15:10:05.313975+0000","last_became_peered":"2024-04-14T15:10:05.313975+0000","last_unstale":"2024-04-14T15:10:42.426656+0000","last_undegraded":"2024-04-14T15:10:42.426656+0000","last_fullsized":"2024-04-14T15:10:42.426656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:10:16.662567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426910+0000","last_change":"2024-04-14T15:10:01.288846+0000","last_active":"2024-04-14T15:10:42.426910+0000","last_peered":"2024-04-14T15:10:42.426910+0000","last_clean":"2024-04-14T15:10:42.426910+0000","last_became_active":"2024-04-14T15:10:01.288167+0000","last_became_peered":"2024-04-14T15:10:01.288167+0000","last_unstale":"2024-04-14T15:10:42.426910+0000","last_undegraded":"2024-04-14T15:10:42.426910+0000","last_fullsized":"2024-04-14T15:10:42.426910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:45:31.589618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505513+0000","last_change":"2024-04-14T15:10:03.303605+0000","last_active":"2024-04-14T15:10:42.505513+0000","last_peered":"2024-04-14T15:10:42.505513+0000","last_clean":"2024-04-14T15:10:42.505513+0000","last_became_active":"2024-04-14T15:10:03.303326+0000","last_became_peered":"2024-04-14T15:10:03.303326+0000","last_unstale":"2024-04-14T15:10:42.505513+0000","last_undegraded":"2024-04-14T15:10:42.505513+0000","last_fullsized":"2024-04-14T15:10:42.505513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:34:53.330974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426439+0000","last_change":"2024-04-14T15:10:07.329923+0000","last_active":"2024-04-14T15:10:42.426439+0000","last_peered":"2024-04-14T15:10:42.426439+0000","last_clean":"2024-04-14T15:10:42.426439+0000","last_became_active":"2024-04-14T15:10:07.329683+0000","last_became_peered":"2024-04-14T15:10:07.329683+0000","last_unstale":"2024-04-14T15:10:42.426439+0000","last_undegraded":"2024-04-14T15:10:42.426439+0000","last_fullsized":"2024-04-14T15:10:42.426439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:56:09.827210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338775+0000","last_change":"2024-04-14T15:10:05.318174+0000","last_active":"2024-04-14T15:10:42.338775+0000","last_peered":"2024-04-14T15:10:42.338775+0000","last_clean":"2024-04-14T15:10:42.338775+0000","last_became_active":"2024-04-14T15:10:05.317854+0000","last_became_peered":"2024-04-14T15:10:05.317854+0000","last_unstale":"2024-04-14T15:10:42.338775+0000","last_undegraded":"2024-04-14T15:10:42.338775+0000","last_fullsized":"2024-04-14T15:10:42.338775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:42:20.184563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426739+0000","last_change":"2024-04-14T15:10:01.294575+0000","last_active":"2024-04-14T15:10:42.426739+0000","last_peered":"2024-04-14T15:10:42.426739+0000","last_clean":"2024-04-14T15:10:42.426739+0000","last_became_active":"2024-04-14T15:10:01.294264+0000","last_became_peered":"2024-04-14T15:10:01.294264+0000","last_unstale":"2024-04-14T15:10:42.426739+0000","last_undegraded":"2024-04-14T15:10:42.426739+0000","last_fullsized":"2024-04-14T15:10:42.426739+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:54:41.706292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505466+0000","last_change":"2024-04-14T15:10:03.306090+0000","last_active":"2024-04-14T15:10:42.505466+0000","last_peered":"2024-04-14T15:10:42.505466+0000","last_clean":"2024-04-14T15:10:42.505466+0000","last_became_active":"2024-04-14T15:10:03.305580+0000","last_became_peered":"2024-04-14T15:10:03.305580+0000","last_unstale":"2024-04-14T15:10:42.505466+0000","last_undegraded":"2024-04-14T15:10:42.505466+0000","last_fullsized":"2024-04-14T15:10:42.505466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:15:37.116468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506009+0000","last_change":"2024-04-14T15:10:07.636906+0000","last_active":"2024-04-14T15:10:42.506009+0000","last_peered":"2024-04-14T15:10:42.506009+0000","last_clean":"2024-04-14T15:10:42.506009+0000","last_became_active":"2024-04-14T15:10:07.636559+0000","last_became_peered":"2024-04-14T15:10:07.636559+0000","last_unstale":"2024-04-14T15:10:42.506009+0000","last_undegraded":"2024-04-14T15:10:42.506009+0000","last_fullsized":"2024-04-14T15:10:42.506009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:58:13.665827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336625+0000","last_change":"2024-04-14T15:10:05.310958+0000","last_active":"2024-04-14T15:10:42.336625+0000","last_peered":"2024-04-14T15:10:42.336625+0000","last_clean":"2024-04-14T15:10:42.336625+0000","last_became_active":"2024-04-14T15:10:05.310586+0000","last_became_peered":"2024-04-14T15:10:05.310586+0000","last_unstale":"2024-04-14T15:10:42.336625+0000","last_undegraded":"2024-04-14T15:10:42.336625+0000","last_fullsized":"2024-04-14T15:10:42.336625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:54:04.107918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.806175+0000","last_change":"2024-04-14T15:10:01.292218+0000","last_active":"2024-04-14T15:10:43.806175+0000","last_peered":"2024-04-14T15:10:43.806175+0000","last_clean":"2024-04-14T15:10:43.806175+0000","last_became_active":"2024-04-14T15:10:01.291952+0000","last_became_peered":"2024-04-14T15:10:01.291952+0000","last_unstale":"2024-04-14T15:10:43.806175+0000","last_undegraded":"2024-04-14T15:10:43.806175+0000","last_fullsized":"2024-04-14T15:10:43.806175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:30:05.912982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339014+0000","last_change":"2024-04-14T15:10:03.298476+0000","last_active":"2024-04-14T15:10:42.339014+0000","last_peered":"2024-04-14T15:10:42.339014+0000","last_clean":"2024-04-14T15:10:42.339014+0000","last_became_active":"2024-04-14T15:10:03.298122+0000","last_became_peered":"2024-04-14T15:10:03.298122+0000","last_unstale":"2024-04-14T15:10:42.339014+0000","last_undegraded":"2024-04-14T15:10:42.339014+0000","last_fullsized":"2024-04-14T15:10:42.339014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:22:18.725525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805385+0000","last_change":"2024-04-14T15:10:07.637163+0000","last_active":"2024-04-14T15:10:43.805385+0000","last_peered":"2024-04-14T15:10:43.805385+0000","last_clean":"2024-04-14T15:10:43.805385+0000","last_became_active":"2024-04-14T15:10:07.636777+0000","last_became_peered":"2024-04-14T15:10:07.636777+0000","last_unstale":"2024-04-14T15:10:43.805385+0000","last_undegraded":"2024-04-14T15:10:43.805385+0000","last_fullsized":"2024-04-14T15:10:43.805385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:24:24.611904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505623+0000","last_change":"2024-04-14T15:10:05.316956+0000","last_active":"2024-04-14T15:10:42.505623+0000","last_peered":"2024-04-14T15:10:42.505623+0000","last_clean":"2024-04-14T15:10:42.505623+0000","last_became_active":"2024-04-14T15:10:05.315776+0000","last_became_peered":"2024-04-14T15:10:05.315776+0000","last_unstale":"2024-04-14T15:10:42.505623+0000","last_undegraded":"2024-04-14T15:10:42.505623+0000","last_fullsized":"2024-04-14T15:10:42.505623+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:29:19.558266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505644+0000","last_change":"2024-04-14T15:10:01.277242+0000","last_active":"2024-04-14T15:10:42.505644+0000","last_peered":"2024-04-14T15:10:42.505644+0000","last_clean":"2024-04-14T15:10:42.505644+0000","last_became_active":"2024-04-14T15:10:01.276939+0000","last_became_peered":"2024-04-14T15:10:01.276939+0000","last_unstale":"2024-04-14T15:10:42.505644+0000","last_undegraded":"2024-04-14T15:10:42.505644+0000","last_fullsized":"2024-04-14T15:10:42.505644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:35:53.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803418+0000","last_change":"2024-04-14T15:10:03.300602+0000","last_active":"2024-04-14T15:10:43.803418+0000","last_peered":"2024-04-14T15:10:43.803418+0000","last_clean":"2024-04-14T15:10:43.803418+0000","last_became_active":"2024-04-14T15:10:03.300270+0000","last_became_peered":"2024-04-14T15:10:03.300270+0000","last_unstale":"2024-04-14T15:10:43.803418+0000","last_undegraded":"2024-04-14T15:10:43.803418+0000","last_fullsized":"2024-04-14T15:10:43.803418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:27:34.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338972+0000","last_change":"2024-04-14T15:10:07.327405+0000","last_active":"2024-04-14T15:10:42.338972+0000","last_peered":"2024-04-14T15:10:42.338972+0000","last_clean":"2024-04-14T15:10:42.338972+0000","last_became_active":"2024-04-14T15:10:07.326994+0000","last_became_peered":"2024-04-14T15:10:07.326994+0000","last_unstale":"2024-04-14T15:10:42.338972+0000","last_undegraded":"2024-04-14T15:10:42.338972+0000","last_fullsized":"2024-04-14T15:10:42.338972+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:00.228979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344606+0000","last_change":"2024-04-14T15:10:05.317523+0000","last_active":"2024-04-14T15:11:16.344606+0000","last_peered":"2024-04-14T15:11:16.344606+0000","last_clean":"2024-04-14T15:11:16.344606+0000","last_became_active":"2024-04-14T15:10:05.317188+0000","last_became_peered":"2024-04-14T15:10:05.317188+0000","last_unstale":"2024-04-14T15:11:16.344606+0000","last_undegraded":"2024-04-14T15:11:16.344606+0000","last_fullsized":"2024-04-14T15:11:16.344606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:45:07.548659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'2","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335833+0000","last_change":"2024-04-14T15:10:01.280775+0000","last_active":"2024-04-14T15:10:42.335833+0000","last_peered":"2024-04-14T15:10:42.335833+0000","last_clean":"2024-04-14T15:10:42.335833+0000","last_became_active":"2024-04-14T15:10:01.280448+0000","last_became_peered":"2024-04-14T15:10:01.280448+0000","last_unstale":"2024-04-14T15:10:42.335833+0000","last_undegraded":"2024-04-14T15:10:42.335833+0000","last_fullsized":"2024-04-14T15:10:42.335833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:30:21.765182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":957,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805737+0000","last_change":"2024-04-14T15:10:03.966220+0000","last_active":"2024-04-14T15:10:43.805737+0000","last_peered":"2024-04-14T15:10:43.805737+0000","last_clean":"2024-04-14T15:10:43.805737+0000","last_became_active":"2024-04-14T15:10:03.965030+0000","last_became_peered":"2024-04-14T15:10:03.965030+0000","last_unstale":"2024-04-14T15:10:43.805737+0000","last_undegraded":"2024-04-14T15:10:43.805737+0000","last_fullsized":"2024-04-14T15:10:43.805737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:15:43.561734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336277+0000","last_change":"2024-04-14T15:10:07.326847+0000","last_active":"2024-04-14T15:10:42.336277+0000","last_peered":"2024-04-14T15:10:42.336277+0000","last_clean":"2024-04-14T15:10:42.336277+0000","last_became_active":"2024-04-14T15:10:07.326530+0000","last_became_peered":"2024-04-14T15:10:07.326530+0000","last_unstale":"2024-04-14T15:10:42.336277+0000","last_undegraded":"2024-04-14T15:10:42.336277+0000","last_fullsized":"2024-04-14T15:10:42.336277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:31:26.084295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.806048+0000","last_change":"2024-04-14T15:10:05.314266+0000","last_active":"2024-04-14T15:10:43.806048+0000","last_peered":"2024-04-14T15:10:43.806048+0000","last_clean":"2024-04-14T15:10:43.806048+0000","last_became_active":"2024-04-14T15:10:05.313939+0000","last_became_peered":"2024-04-14T15:10:05.313939+0000","last_unstale":"2024-04-14T15:10:43.806048+0000","last_undegraded":"2024-04-14T15:10:43.806048+0000","last_fullsized":"2024-04-14T15:10:43.806048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:23.591510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803897+0000","last_change":"2024-04-14T15:10:01.289257+0000","last_active":"2024-04-14T15:10:43.803897+0000","last_peered":"2024-04-14T15:10:43.803897+0000","last_clean":"2024-04-14T15:10:43.803897+0000","last_became_active":"2024-04-14T15:10:01.288972+0000","last_became_peered":"2024-04-14T15:10:01.288972+0000","last_unstale":"2024-04-14T15:10:43.803897+0000","last_undegraded":"2024-04-14T15:10:43.803897+0000","last_fullsized":"2024-04-14T15:10:43.803897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:02:15.791539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802653+0000","last_change":"2024-04-14T15:10:03.299826+0000","last_active":"2024-04-14T15:10:43.802653+0000","last_peered":"2024-04-14T15:10:43.802653+0000","last_clean":"2024-04-14T15:10:43.802653+0000","last_became_active":"2024-04-14T15:10:03.299481+0000","last_became_peered":"2024-04-14T15:10:03.299481+0000","last_unstale":"2024-04-14T15:10:43.802653+0000","last_undegraded":"2024-04-14T15:10:43.802653+0000","last_fullsized":"2024-04-14T15:10:43.802653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:58:57.232199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338718+0000","last_change":"2024-04-14T15:10:07.327521+0000","last_active":"2024-04-14T15:10:42.338718+0000","last_peered":"2024-04-14T15:10:42.338718+0000","last_clean":"2024-04-14T15:10:42.338718+0000","last_became_active":"2024-04-14T15:10:07.327090+0000","last_became_peered":"2024-04-14T15:10:07.327090+0000","last_unstale":"2024-04-14T15:10:42.338718+0000","last_undegraded":"2024-04-14T15:10:42.338718+0000","last_fullsized":"2024-04-14T15:10:42.338718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:51.862824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802847+0000","last_change":"2024-04-14T15:10:05.316779+0000","last_active":"2024-04-14T15:10:43.802847+0000","last_peered":"2024-04-14T15:10:43.802847+0000","last_clean":"2024-04-14T15:10:43.802847+0000","last_became_active":"2024-04-14T15:10:05.316439+0000","last_became_peered":"2024-04-14T15:10:05.316439+0000","last_unstale":"2024-04-14T15:10:43.802847+0000","last_undegraded":"2024-04-14T15:10:43.802847+0000","last_fullsized":"2024-04-14T15:10:43.802847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:36:59.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803554+0000","last_change":"2024-04-14T15:10:01.291409+0000","last_active":"2024-04-14T15:10:43.803554+0000","last_peered":"2024-04-14T15:10:43.803554+0000","last_clean":"2024-04-14T15:10:43.803554+0000","last_became_active":"2024-04-14T15:10:01.291121+0000","last_became_peered":"2024-04-14T15:10:01.291121+0000","last_unstale":"2024-04-14T15:10:43.803554+0000","last_undegraded":"2024-04-14T15:10:43.803554+0000","last_fullsized":"2024-04-14T15:10:43.803554+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:58.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802941+0000","last_change":"2024-04-14T15:10:03.300644+0000","last_active":"2024-04-14T15:10:43.802941+0000","last_peered":"2024-04-14T15:10:43.802941+0000","last_clean":"2024-04-14T15:10:43.802941+0000","last_became_active":"2024-04-14T15:10:03.300369+0000","last_became_peered":"2024-04-14T15:10:03.300369+0000","last_unstale":"2024-04-14T15:10:43.802941+0000","last_undegraded":"2024-04-14T15:10:43.802941+0000","last_fullsized":"2024-04-14T15:10:43.802941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:02:43.981261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804386+0000","last_change":"2024-04-14T15:10:07.639183+0000","last_active":"2024-04-14T15:10:43.804386+0000","last_peered":"2024-04-14T15:10:43.804386+0000","last_clean":"2024-04-14T15:10:43.804386+0000","last_became_active":"2024-04-14T15:10:07.638794+0000","last_became_peered":"2024-04-14T15:10:07.638794+0000","last_unstale":"2024-04-14T15:10:43.804386+0000","last_undegraded":"2024-04-14T15:10:43.804386+0000","last_fullsized":"2024-04-14T15:10:43.804386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:42:43.548702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339150+0000","last_change":"2024-04-14T15:10:05.321002+0000","last_active":"2024-04-14T15:10:42.339150+0000","last_peered":"2024-04-14T15:10:42.339150+0000","last_clean":"2024-04-14T15:10:42.339150+0000","last_became_active":"2024-04-14T15:10:05.320683+0000","last_became_peered":"2024-04-14T15:10:05.320683+0000","last_unstale":"2024-04-14T15:10:42.339150+0000","last_undegraded":"2024-04-14T15:10:42.339150+0000","last_fullsized":"2024-04-14T15:10:42.339150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:29.409123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426979+0000","last_change":"2024-04-14T15:10:01.288082+0000","last_active":"2024-04-14T15:10:42.426979+0000","last_peered":"2024-04-14T15:10:42.426979+0000","last_clean":"2024-04-14T15:10:42.426979+0000","last_became_active":"2024-04-14T15:10:01.287735+0000","last_became_peered":"2024-04-14T15:10:01.287735+0000","last_unstale":"2024-04-14T15:10:42.426979+0000","last_undegraded":"2024-04-14T15:10:42.426979+0000","last_fullsized":"2024-04-14T15:10:42.426979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:00.593000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804095+0000","last_change":"2024-04-14T15:10:03.965980+0000","last_active":"2024-04-14T15:10:43.804095+0000","last_peered":"2024-04-14T15:10:43.804095+0000","last_clean":"2024-04-14T15:10:43.804095+0000","last_became_active":"2024-04-14T15:10:03.965633+0000","last_became_peered":"2024-04-14T15:10:03.965633+0000","last_unstale":"2024-04-14T15:10:43.804095+0000","last_undegraded":"2024-04-14T15:10:43.804095+0000","last_fullsized":"2024-04-14T15:10:43.804095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:49:32.031950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803783+0000","last_change":"2024-04-14T15:10:07.330577+0000","last_active":"2024-04-14T15:10:43.803783+0000","last_peered":"2024-04-14T15:10:43.803783+0000","last_clean":"2024-04-14T15:10:43.803783+0000","last_became_active":"2024-04-14T15:10:07.330215+0000","last_became_peered":"2024-04-14T15:10:07.330215+0000","last_unstale":"2024-04-14T15:10:43.803783+0000","last_undegraded":"2024-04-14T15:10:43.803783+0000","last_fullsized":"2024-04-14T15:10:43.803783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:01:18.146878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804593+0000","last_change":"2024-04-14T15:10:05.312931+0000","last_active":"2024-04-14T15:10:43.804593+0000","last_peered":"2024-04-14T15:10:43.804593+0000","last_clean":"2024-04-14T15:10:43.804593+0000","last_became_active":"2024-04-14T15:10:05.312409+0000","last_became_peered":"2024-04-14T15:10:05.312409+0000","last_unstale":"2024-04-14T15:10:43.804593+0000","last_undegraded":"2024-04-14T15:10:43.804593+0000","last_fullsized":"2024-04-14T15:10:43.804593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:20:35.448982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339250+0000","last_change":"2024-04-14T15:10:01.294947+0000","last_active":"2024-04-14T15:10:42.339250+0000","last_peered":"2024-04-14T15:10:42.339250+0000","last_clean":"2024-04-14T15:10:42.339250+0000","last_became_active":"2024-04-14T15:10:01.294623+0000","last_became_peered":"2024-04-14T15:10:01.294623+0000","last_unstale":"2024-04-14T15:10:42.339250+0000","last_undegraded":"2024-04-14T15:10:42.339250+0000","last_fullsized":"2024-04-14T15:10:42.339250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:42:55.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335999+0000","last_change":"2024-04-14T15:10:03.302392+0000","last_active":"2024-04-14T15:10:42.335999+0000","last_peered":"2024-04-14T15:10:42.335999+0000","last_clean":"2024-04-14T15:10:42.335999+0000","last_became_active":"2024-04-14T15:10:03.301539+0000","last_became_peered":"2024-04-14T15:10:03.301539+0000","last_unstale":"2024-04-14T15:10:42.335999+0000","last_undegraded":"2024-04-14T15:10:42.335999+0000","last_fullsized":"2024-04-14T15:10:42.335999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:44:46.421460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803848+0000","last_change":"2024-04-14T15:10:07.321767+0000","last_active":"2024-04-14T15:10:43.803848+0000","last_peered":"2024-04-14T15:10:43.803848+0000","last_clean":"2024-04-14T15:10:43.803848+0000","last_became_active":"2024-04-14T15:10:07.321406+0000","last_became_peered":"2024-04-14T15:10:07.321406+0000","last_unstale":"2024-04-14T15:10:43.803848+0000","last_undegraded":"2024-04-14T15:10:43.803848+0000","last_fullsized":"2024-04-14T15:10:43.803848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:09:27.865027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335858+0000","last_change":"2024-04-14T15:10:05.317553+0000","last_active":"2024-04-14T15:10:42.335858+0000","last_peered":"2024-04-14T15:10:42.335858+0000","last_clean":"2024-04-14T15:10:42.335858+0000","last_became_active":"2024-04-14T15:10:05.317254+0000","last_became_peered":"2024-04-14T15:10:05.317254+0000","last_unstale":"2024-04-14T15:10:42.335858+0000","last_undegraded":"2024-04-14T15:10:42.335858+0000","last_fullsized":"2024-04-14T15:10:42.335858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:06:43.203130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505865+0000","last_change":"2024-04-14T15:10:01.292065+0000","last_active":"2024-04-14T15:10:42.505865+0000","last_peered":"2024-04-14T15:10:42.505865+0000","last_clean":"2024-04-14T15:10:42.505865+0000","last_became_active":"2024-04-14T15:10:01.291533+0000","last_became_peered":"2024-04-14T15:10:01.291533+0000","last_unstale":"2024-04-14T15:10:42.505865+0000","last_undegraded":"2024-04-14T15:10:42.505865+0000","last_fullsized":"2024-04-14T15:10:42.505865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:23:17.225362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505702+0000","last_change":"2024-04-14T15:10:03.300755+0000","last_active":"2024-04-14T15:10:42.505702+0000","last_peered":"2024-04-14T15:10:42.505702+0000","last_clean":"2024-04-14T15:10:42.505702+0000","last_became_active":"2024-04-14T15:10:03.299638+0000","last_became_peered":"2024-04-14T15:10:03.299638+0000","last_unstale":"2024-04-14T15:10:42.505702+0000","last_undegraded":"2024-04-14T15:10:42.505702+0000","last_fullsized":"2024-04-14T15:10:42.505702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:02:21.922301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805516+0000","last_change":"2024-04-14T15:10:07.323074+0000","last_active":"2024-04-14T15:10:43.805516+0000","last_peered":"2024-04-14T15:10:43.805516+0000","last_clean":"2024-04-14T15:10:43.805516+0000","last_became_active":"2024-04-14T15:10:07.322755+0000","last_became_peered":"2024-04-14T15:10:07.322755+0000","last_unstale":"2024-04-14T15:10:43.805516+0000","last_undegraded":"2024-04-14T15:10:43.805516+0000","last_fullsized":"2024-04-14T15:10:43.805516+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:33:14.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344605+0000","last_change":"2024-04-14T15:10:05.318044+0000","last_active":"2024-04-14T15:11:16.344605+0000","last_peered":"2024-04-14T15:11:16.344605+0000","last_clean":"2024-04-14T15:11:16.344605+0000","last_became_active":"2024-04-14T15:10:05.317538+0000","last_became_peered":"2024-04-14T15:10:05.317538+0000","last_unstale":"2024-04-14T15:11:16.344605+0000","last_undegraded":"2024-04-14T15:11:16.344605+0000","last_fullsized":"2024-04-14T15:11:16.344605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:40:09.782802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338898+0000","last_change":"2024-04-14T15:10:01.285162+0000","last_active":"2024-04-14T15:10:42.338898+0000","last_peered":"2024-04-14T15:10:42.338898+0000","last_clean":"2024-04-14T15:10:42.338898+0000","last_became_active":"2024-04-14T15:10:01.284870+0000","last_became_peered":"2024-04-14T15:10:01.284870+0000","last_unstale":"2024-04-14T15:10:42.338898+0000","last_undegraded":"2024-04-14T15:10:42.338898+0000","last_fullsized":"2024-04-14T15:10:42.338898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:37:29.313852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805034+0000","last_change":"2024-04-14T15:10:03.965870+0000","last_active":"2024-04-14T15:10:43.805034+0000","last_peered":"2024-04-14T15:10:43.805034+0000","last_clean":"2024-04-14T15:10:43.805034+0000","last_became_active":"2024-04-14T15:10:03.965006+0000","last_became_peered":"2024-04-14T15:10:03.965006+0000","last_unstale":"2024-04-14T15:10:43.805034+0000","last_undegraded":"2024-04-14T15:10:43.805034+0000","last_fullsized":"2024-04-14T15:10:43.805034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:40:50.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506147+0000","last_change":"2024-04-14T15:10:07.322845+0000","last_active":"2024-04-14T15:10:42.506147+0000","last_peered":"2024-04-14T15:10:42.506147+0000","last_clean":"2024-04-14T15:10:42.506147+0000","last_became_active":"2024-04-14T15:10:07.322402+0000","last_became_peered":"2024-04-14T15:10:07.322402+0000","last_unstale":"2024-04-14T15:10:42.506147+0000","last_undegraded":"2024-04-14T15:10:42.506147+0000","last_fullsized":"2024-04-14T15:10:42.506147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:36:57.542575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344476+0000","last_change":"2024-04-14T15:10:05.315733+0000","last_active":"2024-04-14T15:11:16.344476+0000","last_peered":"2024-04-14T15:11:16.344476+0000","last_clean":"2024-04-14T15:11:16.344476+0000","last_became_active":"2024-04-14T15:10:05.315375+0000","last_became_peered":"2024-04-14T15:10:05.315375+0000","last_unstale":"2024-04-14T15:11:16.344476+0000","last_undegraded":"2024-04-14T15:11:16.344476+0000","last_fullsized":"2024-04-14T15:11:16.344476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:35:09.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803218+0000","last_change":"2024-04-14T15:10:01.292508+0000","last_active":"2024-04-14T15:10:43.803218+0000","last_peered":"2024-04-14T15:10:43.803218+0000","last_clean":"2024-04-14T15:10:43.803218+0000","last_became_active":"2024-04-14T15:10:01.292048+0000","last_became_peered":"2024-04-14T15:10:01.292048+0000","last_unstale":"2024-04-14T15:10:43.803218+0000","last_undegraded":"2024-04-14T15:10:43.803218+0000","last_fullsized":"2024-04-14T15:10:43.803218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:36:22.472166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427125+0000","last_change":"2024-04-14T15:10:03.965927+0000","last_active":"2024-04-14T15:10:42.427125+0000","last_peered":"2024-04-14T15:10:42.427125+0000","last_clean":"2024-04-14T15:10:42.427125+0000","last_became_active":"2024-04-14T15:10:03.965553+0000","last_became_peered":"2024-04-14T15:10:03.965553+0000","last_unstale":"2024-04-14T15:10:42.427125+0000","last_undegraded":"2024-04-14T15:10:42.427125+0000","last_fullsized":"2024-04-14T15:10:42.427125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:06:55.504599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339383+0000","last_change":"2024-04-14T15:10:07.321561+0000","last_active":"2024-04-14T15:10:42.339383+0000","last_peered":"2024-04-14T15:10:42.339383+0000","last_clean":"2024-04-14T15:10:42.339383+0000","last_became_active":"2024-04-14T15:10:07.320311+0000","last_became_peered":"2024-04-14T15:10:07.320311+0000","last_unstale":"2024-04-14T15:10:42.339383+0000","last_undegraded":"2024-04-14T15:10:42.339383+0000","last_fullsized":"2024-04-14T15:10:42.339383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:09.072546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805516+0000","last_change":"2024-04-14T15:10:05.316174+0000","last_active":"2024-04-14T15:10:43.805516+0000","last_peered":"2024-04-14T15:10:43.805516+0000","last_clean":"2024-04-14T15:10:43.805516+0000","last_became_active":"2024-04-14T15:10:05.315807+0000","last_became_peered":"2024-04-14T15:10:05.315807+0000","last_unstale":"2024-04-14T15:10:43.805516+0000","last_undegraded":"2024-04-14T15:10:43.805516+0000","last_fullsized":"2024-04-14T15:10:43.805516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:43:37.742051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339466+0000","last_change":"2024-04-14T15:10:01.289169+0000","last_active":"2024-04-14T15:10:42.339466+0000","last_peered":"2024-04-14T15:10:42.339466+0000","last_clean":"2024-04-14T15:10:42.339466+0000","last_became_active":"2024-04-14T15:10:01.288924+0000","last_became_peered":"2024-04-14T15:10:01.288924+0000","last_unstale":"2024-04-14T15:10:42.339466+0000","last_undegraded":"2024-04-14T15:10:42.339466+0000","last_fullsized":"2024-04-14T15:10:42.339466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:25:27.657428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805864+0000","last_change":"2024-04-14T15:10:03.966576+0000","last_active":"2024-04-14T15:10:43.805864+0000","last_peered":"2024-04-14T15:10:43.805864+0000","last_clean":"2024-04-14T15:10:43.805864+0000","last_became_active":"2024-04-14T15:10:03.965869+0000","last_became_peered":"2024-04-14T15:10:03.965869+0000","last_unstale":"2024-04-14T15:10:43.805864+0000","last_undegraded":"2024-04-14T15:10:43.805864+0000","last_fullsized":"2024-04-14T15:10:43.805864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:00:04.336610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339141+0000","last_change":"2024-04-14T15:10:07.639897+0000","last_active":"2024-04-14T15:10:42.339141+0000","last_peered":"2024-04-14T15:10:42.339141+0000","last_clean":"2024-04-14T15:10:42.339141+0000","last_became_active":"2024-04-14T15:10:07.639314+0000","last_became_peered":"2024-04-14T15:10:07.639314+0000","last_unstale":"2024-04-14T15:10:42.339141+0000","last_undegraded":"2024-04-14T15:10:42.339141+0000","last_fullsized":"2024-04-14T15:10:42.339141+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:50:22.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426982+0000","last_change":"2024-04-14T15:10:05.317039+0000","last_active":"2024-04-14T15:10:42.426982+0000","last_peered":"2024-04-14T15:10:42.426982+0000","last_clean":"2024-04-14T15:10:42.426982+0000","last_became_active":"2024-04-14T15:10:05.316733+0000","last_became_peered":"2024-04-14T15:10:05.316733+0000","last_unstale":"2024-04-14T15:10:42.426982+0000","last_undegraded":"2024-04-14T15:10:42.426982+0000","last_fullsized":"2024-04-14T15:10:42.426982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:38:55.116276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802736+0000","last_change":"2024-04-14T15:10:01.293838+0000","last_active":"2024-04-14T15:10:43.802736+0000","last_peered":"2024-04-14T15:10:43.802736+0000","last_clean":"2024-04-14T15:10:43.802736+0000","last_became_active":"2024-04-14T15:10:01.293517+0000","last_became_peered":"2024-04-14T15:10:01.293517+0000","last_unstale":"2024-04-14T15:10:43.802736+0000","last_undegraded":"2024-04-14T15:10:43.802736+0000","last_fullsized":"2024-04-14T15:10:43.802736+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:24:27.988036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805559+0000","last_change":"2024-04-14T15:10:03.965531+0000","last_active":"2024-04-14T15:10:43.805559+0000","last_peered":"2024-04-14T15:10:43.805559+0000","last_clean":"2024-04-14T15:10:43.805559+0000","last_became_active":"2024-04-14T15:10:03.965047+0000","last_became_peered":"2024-04-14T15:10:03.965047+0000","last_unstale":"2024-04-14T15:10:43.805559+0000","last_undegraded":"2024-04-14T15:10:43.805559+0000","last_fullsized":"2024-04-14T15:10:43.805559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:32:32.285818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426723+0000","last_change":"2024-04-14T15:10:07.637285+0000","last_active":"2024-04-14T15:10:42.426723+0000","last_peered":"2024-04-14T15:10:42.426723+0000","last_clean":"2024-04-14T15:10:42.426723+0000","last_became_active":"2024-04-14T15:10:07.636996+0000","last_became_peered":"2024-04-14T15:10:07.636996+0000","last_unstale":"2024-04-14T15:10:42.426723+0000","last_undegraded":"2024-04-14T15:10:42.426723+0000","last_fullsized":"2024-04-14T15:10:42.426723+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:30:23.406344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339659+0000","last_change":"2024-04-14T15:10:01.295413+0000","last_active":"2024-04-14T15:10:42.339659+0000","last_peered":"2024-04-14T15:10:42.339659+0000","last_clean":"2024-04-14T15:10:42.339659+0000","last_became_active":"2024-04-14T15:10:01.295169+0000","last_became_peered":"2024-04-14T15:10:01.295169+0000","last_unstale":"2024-04-14T15:10:42.339659+0000","last_undegraded":"2024-04-14T15:10:42.339659+0000","last_fullsized":"2024-04-14T15:10:42.339659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:47:48.534661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426471+0000","last_change":"2024-04-14T15:10:03.965452+0000","last_active":"2024-04-14T15:10:42.426471+0000","last_peered":"2024-04-14T15:10:42.426471+0000","last_clean":"2024-04-14T15:10:42.426471+0000","last_became_active":"2024-04-14T15:10:03.965017+0000","last_became_peered":"2024-04-14T15:10:03.965017+0000","last_unstale":"2024-04-14T15:10:42.426471+0000","last_undegraded":"2024-04-14T15:10:42.426471+0000","last_fullsized":"2024-04-14T15:10:42.426471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:18.308142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344612+0000","last_change":"2024-04-14T15:10:05.317320+0000","last_active":"2024-04-14T15:11:16.344612+0000","last_peered":"2024-04-14T15:11:16.344612+0000","last_clean":"2024-04-14T15:11:16.344612+0000","last_became_active":"2024-04-14T15:10:05.317004+0000","last_became_peered":"2024-04-14T15:10:05.317004+0000","last_unstale":"2024-04-14T15:11:16.344612+0000","last_undegraded":"2024-04-14T15:11:16.344612+0000","last_fullsized":"2024-04-14T15:11:16.344612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:49:14.442926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804483+0000","last_change":"2024-04-14T15:10:07.323919+0000","last_active":"2024-04-14T15:10:43.804483+0000","last_peered":"2024-04-14T15:10:43.804483+0000","last_clean":"2024-04-14T15:10:43.804483+0000","last_became_active":"2024-04-14T15:10:07.323558+0000","last_became_peered":"2024-04-14T15:10:07.323558+0000","last_unstale":"2024-04-14T15:10:43.804483+0000","last_undegraded":"2024-04-14T15:10:43.804483+0000","last_fullsized":"2024-04-14T15:10:43.804483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:19:05.570992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339384+0000","last_change":"2024-04-14T15:10:01.286196+0000","last_active":"2024-04-14T15:10:42.339384+0000","last_peered":"2024-04-14T15:10:42.339384+0000","last_clean":"2024-04-14T15:10:42.339384+0000","last_became_active":"2024-04-14T15:10:01.285912+0000","last_became_peered":"2024-04-14T15:10:01.285912+0000","last_unstale":"2024-04-14T15:10:42.339384+0000","last_undegraded":"2024-04-14T15:10:42.339384+0000","last_fullsized":"2024-04-14T15:10:42.339384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:25:42.738433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505605+0000","last_change":"2024-04-14T15:10:03.299918+0000","last_active":"2024-04-14T15:10:42.505605+0000","last_peered":"2024-04-14T15:10:42.505605+0000","last_clean":"2024-04-14T15:10:42.505605+0000","last_became_active":"2024-04-14T15:10:03.299558+0000","last_became_peered":"2024-04-14T15:10:03.299558+0000","last_unstale":"2024-04-14T15:10:42.505605+0000","last_undegraded":"2024-04-14T15:10:42.505605+0000","last_fullsized":"2024-04-14T15:10:42.505605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:59.005676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338377+0000","last_change":"2024-04-14T15:10:05.309890+0000","last_active":"2024-04-14T15:10:42.338377+0000","last_peered":"2024-04-14T15:10:42.338377+0000","last_clean":"2024-04-14T15:10:42.338377+0000","last_became_active":"2024-04-14T15:10:05.309549+0000","last_became_peered":"2024-04-14T15:10:05.309549+0000","last_unstale":"2024-04-14T15:10:42.338377+0000","last_undegraded":"2024-04-14T15:10:42.338377+0000","last_fullsized":"2024-04-14T15:10:42.338377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:33:49.020700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338599+0000","last_change":"2024-04-14T15:10:07.321998+0000","last_active":"2024-04-14T15:10:42.338599+0000","last_peered":"2024-04-14T15:10:42.338599+0000","last_clean":"2024-04-14T15:10:42.338599+0000","last_became_active":"2024-04-14T15:10:07.321680+0000","last_became_peered":"2024-04-14T15:10:07.321680+0000","last_unstale":"2024-04-14T15:10:42.338599+0000","last_undegraded":"2024-04-14T15:10:42.338599+0000","last_fullsized":"2024-04-14T15:10:42.338599+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:37:08.760904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339122+0000","last_change":"2024-04-14T15:10:05.314886+0000","last_active":"2024-04-14T15:10:42.339122+0000","last_peered":"2024-04-14T15:10:42.339122+0000","last_clean":"2024-04-14T15:10:42.339122+0000","last_became_active":"2024-04-14T15:10:05.314443+0000","last_became_peered":"2024-04-14T15:10:05.314443+0000","last_unstale":"2024-04-14T15:10:42.339122+0000","last_undegraded":"2024-04-14T15:10:42.339122+0000","last_fullsized":"2024-04-14T15:10:42.339122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:14:28.738736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339350+0000","last_change":"2024-04-14T15:10:01.295011+0000","last_active":"2024-04-14T15:10:42.339350+0000","last_peered":"2024-04-14T15:10:42.339350+0000","last_clean":"2024-04-14T15:10:42.339350+0000","last_became_active":"2024-04-14T15:10:01.294717+0000","last_became_peered":"2024-04-14T15:10:01.294717+0000","last_unstale":"2024-04-14T15:10:42.339350+0000","last_undegraded":"2024-04-14T15:10:42.339350+0000","last_fullsized":"2024-04-14T15:10:42.339350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:29:40.393469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803196+0000","last_change":"2024-04-14T15:10:03.965210+0000","last_active":"2024-04-14T15:10:43.803196+0000","last_peered":"2024-04-14T15:10:43.803196+0000","last_clean":"2024-04-14T15:10:43.803196+0000","last_became_active":"2024-04-14T15:10:03.964870+0000","last_became_peered":"2024-04-14T15:10:03.964870+0000","last_unstale":"2024-04-14T15:10:43.803196+0000","last_undegraded":"2024-04-14T15:10:43.803196+0000","last_fullsized":"2024-04-14T15:10:43.803196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:01:52.144341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803967+0000","last_change":"2024-04-14T15:10:07.326689+0000","last_active":"2024-04-14T15:10:43.803967+0000","last_peered":"2024-04-14T15:10:43.803967+0000","last_clean":"2024-04-14T15:10:43.803967+0000","last_became_active":"2024-04-14T15:10:07.326206+0000","last_became_peered":"2024-04-14T15:10:07.326206+0000","last_unstale":"2024-04-14T15:10:43.803967+0000","last_undegraded":"2024-04-14T15:10:43.803967+0000","last_fullsized":"2024-04-14T15:10:43.803967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:37:00.130858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338804+0000","last_change":"2024-04-14T15:10:05.312258+0000","last_active":"2024-04-14T15:10:42.338804+0000","last_peered":"2024-04-14T15:10:42.338804+0000","last_clean":"2024-04-14T15:10:42.338804+0000","last_became_active":"2024-04-14T15:10:05.311943+0000","last_became_peered":"2024-04-14T15:10:05.311943+0000","last_unstale":"2024-04-14T15:10:42.338804+0000","last_undegraded":"2024-04-14T15:10:42.338804+0000","last_fullsized":"2024-04-14T15:10:42.338804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:14:45.667468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426810+0000","last_change":"2024-04-14T15:10:01.290547+0000","last_active":"2024-04-14T15:10:42.426810+0000","last_peered":"2024-04-14T15:10:42.426810+0000","last_clean":"2024-04-14T15:10:42.426810+0000","last_became_active":"2024-04-14T15:10:01.290140+0000","last_became_peered":"2024-04-14T15:10:01.290140+0000","last_unstale":"2024-04-14T15:10:42.426810+0000","last_undegraded":"2024-04-14T15:10:42.426810+0000","last_fullsized":"2024-04-14T15:10:42.426810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:13.843146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803366+0000","last_change":"2024-04-14T15:10:03.303911+0000","last_active":"2024-04-14T15:10:43.803366+0000","last_peered":"2024-04-14T15:10:43.803366+0000","last_clean":"2024-04-14T15:10:43.803366+0000","last_became_active":"2024-04-14T15:10:03.303344+0000","last_became_peered":"2024-04-14T15:10:03.303344+0000","last_unstale":"2024-04-14T15:10:43.803366+0000","last_undegraded":"2024-04-14T15:10:43.803366+0000","last_fullsized":"2024-04-14T15:10:43.803366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:16:56.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804561+0000","last_change":"2024-04-14T15:10:07.319951+0000","last_active":"2024-04-14T15:10:43.804561+0000","last_peered":"2024-04-14T15:10:43.804561+0000","last_clean":"2024-04-14T15:10:43.804561+0000","last_became_active":"2024-04-14T15:10:07.319553+0000","last_became_peered":"2024-04-14T15:10:07.319553+0000","last_unstale":"2024-04-14T15:10:43.804561+0000","last_undegraded":"2024-04-14T15:10:43.804561+0000","last_fullsized":"2024-04-14T15:10:43.804561+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:49:18.780378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426552+0000","last_change":"2024-04-14T15:10:05.310230+0000","last_active":"2024-04-14T15:10:42.426552+0000","last_peered":"2024-04-14T15:10:42.426552+0000","last_clean":"2024-04-14T15:10:42.426552+0000","last_became_active":"2024-04-14T15:10:01.287454+0000","last_became_peered":"2024-04-14T15:10:01.287454+0000","last_unstale":"2024-04-14T15:10:42.426552+0000","last_undegraded":"2024-04-14T15:10:42.426552+0000","last_fullsized":"2024-04-14T15:10:42.426552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:25:28.200138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020339440000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505521+0000","last_change":"2024-04-14T15:10:01.285979+0000","last_active":"2024-04-14T15:10:42.505521+0000","last_peered":"2024-04-14T15:10:42.505521+0000","last_clean":"2024-04-14T15:10:42.505521+0000","last_became_active":"2024-04-14T15:10:01.285625+0000","last_became_peered":"2024-04-14T15:10:01.285625+0000","last_unstale":"2024-04-14T15:10:42.505521+0000","last_undegraded":"2024-04-14T15:10:42.505521+0000","last_fullsized":"2024-04-14T15:10:42.505521+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:00.047342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":90,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344451+0000","last_change":"2024-04-14T15:10:03.302028+0000","last_active":"2024-04-14T15:11:16.344451+0000","last_peered":"2024-04-14T15:11:16.344451+0000","last_clean":"2024-04-14T15:11:16.344451+0000","last_became_active":"2024-04-14T15:10:03.301541+0000","last_became_peered":"2024-04-14T15:10:03.301541+0000","last_unstale":"2024-04-14T15:11:16.344451+0000","last_undegraded":"2024-04-14T15:11:16.344451+0000","last_fullsized":"2024-04-14T15:11:16.344451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:47:21.989515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804260+0000","last_change":"2024-04-14T15:10:05.314336+0000","last_active":"2024-04-14T15:10:43.804260+0000","last_peered":"2024-04-14T15:10:43.804260+0000","last_clean":"2024-04-14T15:10:43.804260+0000","last_became_active":"2024-04-14T15:10:05.314003+0000","last_became_peered":"2024-04-14T15:10:05.314003+0000","last_unstale":"2024-04-14T15:10:43.804260+0000","last_undegraded":"2024-04-14T15:10:43.804260+0000","last_fullsized":"2024-04-14T15:10:43.804260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:31:52.222651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339439+0000","last_change":"2024-04-14T15:10:07.637085+0000","last_active":"2024-04-14T15:10:42.339439+0000","last_peered":"2024-04-14T15:10:42.339439+0000","last_clean":"2024-04-14T15:10:42.339439+0000","last_became_active":"2024-04-14T15:10:07.636768+0000","last_became_peered":"2024-04-14T15:10:07.636768+0000","last_unstale":"2024-04-14T15:10:42.339439+0000","last_undegraded":"2024-04-14T15:10:42.339439+0000","last_fullsized":"2024-04-14T15:10:42.339439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:10:35.431103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335782+0000","last_change":"2024-04-14T15:10:01.285910+0000","last_active":"2024-04-14T15:10:42.335782+0000","last_peered":"2024-04-14T15:10:42.335782+0000","last_clean":"2024-04-14T15:10:42.335782+0000","last_became_active":"2024-04-14T15:10:01.285450+0000","last_became_peered":"2024-04-14T15:10:01.285450+0000","last_unstale":"2024-04-14T15:10:42.335782+0000","last_undegraded":"2024-04-14T15:10:42.335782+0000","last_fullsized":"2024-04-14T15:10:42.335782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:31:51.247210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338658+0000","last_change":"2024-04-14T15:10:05.302122+0000","last_active":"2024-04-14T15:10:42.338658+0000","last_peered":"2024-04-14T15:10:42.338658+0000","last_clean":"2024-04-14T15:10:42.338658+0000","last_became_active":"2024-04-14T15:10:01.284051+0000","last_became_peered":"2024-04-14T15:10:01.284051+0000","last_unstale":"2024-04-14T15:10:42.338658+0000","last_undegraded":"2024-04-14T15:10:42.338658+0000","last_fullsized":"2024-04-14T15:10:42.338658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:24.016649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086327099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":52,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336938+0000","last_change":"2024-04-14T15:10:03.299762+0000","last_active":"2024-04-14T15:10:42.336938+0000","last_peered":"2024-04-14T15:10:42.336938+0000","last_clean":"2024-04-14T15:10:42.336938+0000","last_became_active":"2024-04-14T15:10:03.299269+0000","last_became_peered":"2024-04-14T15:10:03.299269+0000","last_unstale":"2024-04-14T15:10:42.336938+0000","last_undegraded":"2024-04-14T15:10:42.336938+0000","last_fullsized":"2024-04-14T15:10:42.336938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:36:41.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339511+0000","last_change":"2024-04-14T15:10:05.316383+0000","last_active":"2024-04-14T15:10:42.339511+0000","last_peered":"2024-04-14T15:10:42.339511+0000","last_clean":"2024-04-14T15:10:42.339511+0000","last_became_active":"2024-04-14T15:10:05.316049+0000","last_became_peered":"2024-04-14T15:10:05.316049+0000","last_unstale":"2024-04-14T15:10:42.339511+0000","last_undegraded":"2024-04-14T15:10:42.339511+0000","last_fullsized":"2024-04-14T15:10:42.339511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:45:29.783005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427054+0000","last_change":"2024-04-14T15:10:07.322544+0000","last_active":"2024-04-14T15:10:42.427054+0000","last_peered":"2024-04-14T15:10:42.427054+0000","last_clean":"2024-04-14T15:10:42.427054+0000","last_became_active":"2024-04-14T15:10:07.322208+0000","last_became_peered":"2024-04-14T15:10:07.322208+0000","last_unstale":"2024-04-14T15:10:42.427054+0000","last_undegraded":"2024-04-14T15:10:42.427054+0000","last_fullsized":"2024-04-14T15:10:42.427054+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:49:13.037785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":268,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:44.420371+0000","last_change":"2024-04-14T15:09:52.891958+0000","last_active":"2024-04-14T15:10:44.420371+0000","last_peered":"2024-04-14T15:10:44.420371+0000","last_clean":"2024-04-14T15:10:44.420371+0000","last_became_active":"2024-04-14T15:09:52.891540+0000","last_became_peered":"2024-04-14T15:09:52.891540+0000","last_unstale":"2024-04-14T15:10:44.420371+0000","last_undegraded":"2024-04-14T15:10:44.420371+0000","last_fullsized":"2024-04-14T15:10:44.420371+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:07:49.498054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:07:49.498054+0000","last_clean_scrub_stamp":"2024-04-14T15:07:49.498054+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:20:20.335562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804400+0000","last_change":"2024-04-14T15:10:01.289421+0000","last_active":"2024-04-14T15:10:43.804400+0000","last_peered":"2024-04-14T15:10:43.804400+0000","last_clean":"2024-04-14T15:10:43.804400+0000","last_became_active":"2024-04-14T15:10:01.287764+0000","last_became_peered":"2024-04-14T15:10:01.287764+0000","last_unstale":"2024-04-14T15:10:43.804400+0000","last_undegraded":"2024-04-14T15:10:43.804400+0000","last_fullsized":"2024-04-14T15:10:43.804400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:29:21.212035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344605+0000","last_change":"2024-04-14T15:10:03.965170+0000","last_active":"2024-04-14T15:11:16.344605+0000","last_peered":"2024-04-14T15:11:16.344605+0000","last_clean":"2024-04-14T15:11:16.344605+0000","last_became_active":"2024-04-14T15:10:03.964799+0000","last_became_peered":"2024-04-14T15:10:03.964799+0000","last_unstale":"2024-04-14T15:11:16.344605+0000","last_undegraded":"2024-04-14T15:11:16.344605+0000","last_fullsized":"2024-04-14T15:11:16.344605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:02:54.515909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339083+0000","last_change":"2024-04-14T15:10:05.318055+0000","last_active":"2024-04-14T15:10:42.339083+0000","last_peered":"2024-04-14T15:10:42.339083+0000","last_clean":"2024-04-14T15:10:42.339083+0000","last_became_active":"2024-04-14T15:10:05.317719+0000","last_became_peered":"2024-04-14T15:10:05.317719+0000","last_unstale":"2024-04-14T15:10:42.339083+0000","last_undegraded":"2024-04-14T15:10:42.339083+0000","last_fullsized":"2024-04-14T15:10:42.339083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:21:16.451753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335517+0000","last_change":"2024-04-14T15:10:07.331183+0000","last_active":"2024-04-14T15:10:42.335517+0000","last_peered":"2024-04-14T15:10:42.335517+0000","last_clean":"2024-04-14T15:10:42.335517+0000","last_became_active":"2024-04-14T15:10:07.330854+0000","last_became_peered":"2024-04-14T15:10:07.330854+0000","last_unstale":"2024-04-14T15:10:42.335517+0000","last_undegraded":"2024-04-14T15:10:42.335517+0000","last_fullsized":"2024-04-14T15:10:42.335517+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:37:30.621357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804562+0000","last_change":"2024-04-14T15:10:01.291751+0000","last_active":"2024-04-14T15:10:43.804562+0000","last_peered":"2024-04-14T15:10:43.804562+0000","last_clean":"2024-04-14T15:10:43.804562+0000","last_became_active":"2024-04-14T15:10:01.291431+0000","last_became_peered":"2024-04-14T15:10:01.291431+0000","last_unstale":"2024-04-14T15:10:43.804562+0000","last_undegraded":"2024-04-14T15:10:43.804562+0000","last_fullsized":"2024-04-14T15:10:43.804562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:45.110783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:21.669431+0000","last_change":"2024-04-14T15:10:05.309258+0000","last_active":"2024-04-14T15:11:21.669431+0000","last_peered":"2024-04-14T15:11:21.669431+0000","last_clean":"2024-04-14T15:11:21.669431+0000","last_became_active":"2024-04-14T15:10:01.290632+0000","last_became_peered":"2024-04-14T15:10:01.290632+0000","last_unstale":"2024-04-14T15:11:21.669431+0000","last_undegraded":"2024-04-14T15:11:21.669431+0000","last_fullsized":"2024-04-14T15:11:21.669431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:02.840870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043836199999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805335+0000","last_change":"2024-04-14T15:10:03.300802+0000","last_active":"2024-04-14T15:10:43.805335+0000","last_peered":"2024-04-14T15:10:43.805335+0000","last_clean":"2024-04-14T15:10:43.805335+0000","last_became_active":"2024-04-14T15:10:03.300483+0000","last_became_peered":"2024-04-14T15:10:03.300483+0000","last_unstale":"2024-04-14T15:10:43.805335+0000","last_undegraded":"2024-04-14T15:10:43.805335+0000","last_fullsized":"2024-04-14T15:10:43.805335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:18:51.955792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427473+0000","last_change":"2024-04-14T15:10:05.316548+0000","last_active":"2024-04-14T15:10:42.427473+0000","last_peered":"2024-04-14T15:10:42.427473+0000","last_clean":"2024-04-14T15:10:42.427473+0000","last_became_active":"2024-04-14T15:10:05.316218+0000","last_became_peered":"2024-04-14T15:10:05.316218+0000","last_unstale":"2024-04-14T15:10:42.427473+0000","last_undegraded":"2024-04-14T15:10:42.427473+0000","last_fullsized":"2024-04-14T15:10:42.427473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:17:18.940333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335523+0000","last_change":"2024-04-14T15:10:07.637225+0000","last_active":"2024-04-14T15:10:42.335523+0000","last_peered":"2024-04-14T15:10:42.335523+0000","last_clean":"2024-04-14T15:10:42.335523+0000","last_became_active":"2024-04-14T15:10:07.636840+0000","last_became_peered":"2024-04-14T15:10:07.636840+0000","last_unstale":"2024-04-14T15:10:42.335523+0000","last_undegraded":"2024-04-14T15:10:42.335523+0000","last_fullsized":"2024-04-14T15:10:42.335523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:08.889538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336182+0000","last_change":"2024-04-14T15:10:01.283024+0000","last_active":"2024-04-14T15:10:42.336182+0000","last_peered":"2024-04-14T15:10:42.336182+0000","last_clean":"2024-04-14T15:10:42.336182+0000","last_became_active":"2024-04-14T15:10:01.282699+0000","last_became_peered":"2024-04-14T15:10:01.282699+0000","last_unstale":"2024-04-14T15:10:42.336182+0000","last_undegraded":"2024-04-14T15:10:42.336182+0000","last_fullsized":"2024-04-14T15:10:42.336182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:40:30.016178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":61,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803930+0000","last_change":"2024-04-14T15:10:03.965436+0000","last_active":"2024-04-14T15:10:43.803930+0000","last_peered":"2024-04-14T15:10:43.803930+0000","last_clean":"2024-04-14T15:10:43.803930+0000","last_became_active":"2024-04-14T15:10:03.965035+0000","last_became_peered":"2024-04-14T15:10:03.965035+0000","last_unstale":"2024-04-14T15:10:43.803930+0000","last_undegraded":"2024-04-14T15:10:43.803930+0000","last_fullsized":"2024-04-14T15:10:43.803930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:03:59.112891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803197+0000","last_change":"2024-04-14T15:10:05.311617+0000","last_active":"2024-04-14T15:10:43.803197+0000","last_peered":"2024-04-14T15:10:43.803197+0000","last_clean":"2024-04-14T15:10:43.803197+0000","last_became_active":"2024-04-14T15:10:05.311258+0000","last_became_peered":"2024-04-14T15:10:05.311258+0000","last_unstale":"2024-04-14T15:10:43.803197+0000","last_undegraded":"2024-04-14T15:10:43.803197+0000","last_fullsized":"2024-04-14T15:10:43.803197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:11:26.931324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803906+0000","last_change":"2024-04-14T15:10:07.326925+0000","last_active":"2024-04-14T15:10:43.803906+0000","last_peered":"2024-04-14T15:10:43.803906+0000","last_clean":"2024-04-14T15:10:43.803906+0000","last_became_active":"2024-04-14T15:10:07.326573+0000","last_became_peered":"2024-04-14T15:10:07.326573+0000","last_unstale":"2024-04-14T15:10:43.803906+0000","last_undegraded":"2024-04-14T15:10:43.803906+0000","last_fullsized":"2024-04-14T15:10:43.803906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:04:29.291846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427022+0000","last_change":"2024-04-14T15:10:01.291008+0000","last_active":"2024-04-14T15:10:42.427022+0000","last_peered":"2024-04-14T15:10:42.427022+0000","last_clean":"2024-04-14T15:10:42.427022+0000","last_became_active":"2024-04-14T15:10:01.289754+0000","last_became_peered":"2024-04-14T15:10:01.289754+0000","last_unstale":"2024-04-14T15:10:42.427022+0000","last_undegraded":"2024-04-14T15:10:42.427022+0000","last_fullsized":"2024-04-14T15:10:42.427022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:07:50.687334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335967+0000","last_change":"2024-04-14T15:10:03.301866+0000","last_active":"2024-04-14T15:10:42.335967+0000","last_peered":"2024-04-14T15:10:42.335967+0000","last_clean":"2024-04-14T15:10:42.335967+0000","last_became_active":"2024-04-14T15:10:03.299690+0000","last_became_peered":"2024-04-14T15:10:03.299690+0000","last_unstale":"2024-04-14T15:10:42.335967+0000","last_undegraded":"2024-04-14T15:10:42.335967+0000","last_fullsized":"2024-04-14T15:10:42.335967+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:28:13.401054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344458+0000","last_change":"2024-04-14T15:10:05.312319+0000","last_active":"2024-04-14T15:11:16.344458+0000","last_peered":"2024-04-14T15:11:16.344458+0000","last_clean":"2024-04-14T15:11:16.344458+0000","last_became_active":"2024-04-14T15:10:05.311910+0000","last_became_peered":"2024-04-14T15:10:05.311910+0000","last_unstale":"2024-04-14T15:11:16.344458+0000","last_undegraded":"2024-04-14T15:11:16.344458+0000","last_fullsized":"2024-04-14T15:11:16.344458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:46:38.273376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339013+0000","last_change":"2024-04-14T15:10:07.639889+0000","last_active":"2024-04-14T15:10:42.339013+0000","last_peered":"2024-04-14T15:10:42.339013+0000","last_clean":"2024-04-14T15:10:42.339013+0000","last_became_active":"2024-04-14T15:10:07.639568+0000","last_became_peered":"2024-04-14T15:10:07.639568+0000","last_unstale":"2024-04-14T15:10:42.339013+0000","last_undegraded":"2024-04-14T15:10:42.339013+0000","last_fullsized":"2024-04-14T15:10:42.339013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:44:17.822595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805627+0000","last_change":"2024-04-14T15:10:01.285916+0000","last_active":"2024-04-14T15:10:43.805627+0000","last_peered":"2024-04-14T15:10:43.805627+0000","last_clean":"2024-04-14T15:10:43.805627+0000","last_became_active":"2024-04-14T15:10:01.285596+0000","last_became_peered":"2024-04-14T15:10:01.285596+0000","last_unstale":"2024-04-14T15:10:43.805627+0000","last_undegraded":"2024-04-14T15:10:43.805627+0000","last_fullsized":"2024-04-14T15:10:43.805627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:17.934968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505453+0000","last_change":"2024-04-14T15:10:03.299486+0000","last_active":"2024-04-14T15:10:42.505453+0000","last_peered":"2024-04-14T15:10:42.505453+0000","last_clean":"2024-04-14T15:10:42.505453+0000","last_became_active":"2024-04-14T15:10:03.299136+0000","last_became_peered":"2024-04-14T15:10:03.299136+0000","last_unstale":"2024-04-14T15:10:42.505453+0000","last_undegraded":"2024-04-14T15:10:42.505453+0000","last_fullsized":"2024-04-14T15:10:42.505453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:42:31.753611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344449+0000","last_change":"2024-04-14T15:10:05.317136+0000","last_active":"2024-04-14T15:11:16.344449+0000","last_peered":"2024-04-14T15:11:16.344449+0000","last_clean":"2024-04-14T15:11:16.344449+0000","last_became_active":"2024-04-14T15:10:05.316824+0000","last_became_peered":"2024-04-14T15:10:05.316824+0000","last_unstale":"2024-04-14T15:11:16.344449+0000","last_undegraded":"2024-04-14T15:11:16.344449+0000","last_fullsized":"2024-04-14T15:11:16.344449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:48:05.709161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505410+0000","last_change":"2024-04-14T15:10:07.316986+0000","last_active":"2024-04-14T15:10:42.505410+0000","last_peered":"2024-04-14T15:10:42.505410+0000","last_clean":"2024-04-14T15:10:42.505410+0000","last_became_active":"2024-04-14T15:10:07.316647+0000","last_became_peered":"2024-04-14T15:10:07.316647+0000","last_unstale":"2024-04-14T15:10:42.505410+0000","last_undegraded":"2024-04-14T15:10:42.505410+0000","last_fullsized":"2024-04-14T15:10:42.505410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:35:59.372100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805459+0000","last_change":"2024-04-14T15:10:01.289161+0000","last_active":"2024-04-14T15:10:43.805459+0000","last_peered":"2024-04-14T15:10:43.805459+0000","last_clean":"2024-04-14T15:10:43.805459+0000","last_became_active":"2024-04-14T15:10:01.288811+0000","last_became_peered":"2024-04-14T15:10:01.288811+0000","last_unstale":"2024-04-14T15:10:43.805459+0000","last_undegraded":"2024-04-14T15:10:43.805459+0000","last_fullsized":"2024-04-14T15:10:43.805459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:05:59.930810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804114+0000","last_change":"2024-04-14T15:10:03.965610+0000","last_active":"2024-04-14T15:10:43.804114+0000","last_peered":"2024-04-14T15:10:43.804114+0000","last_clean":"2024-04-14T15:10:43.804114+0000","last_became_active":"2024-04-14T15:10:03.965206+0000","last_became_peered":"2024-04-14T15:10:03.965206+0000","last_unstale":"2024-04-14T15:10:43.804114+0000","last_undegraded":"2024-04-14T15:10:43.804114+0000","last_fullsized":"2024-04-14T15:10:43.804114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:06:23.824599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505320+0000","last_change":"2024-04-14T15:10:05.318945+0000","last_active":"2024-04-14T15:10:42.505320+0000","last_peered":"2024-04-14T15:10:42.505320+0000","last_clean":"2024-04-14T15:10:42.505320+0000","last_became_active":"2024-04-14T15:10:05.317388+0000","last_became_peered":"2024-04-14T15:10:05.317388+0000","last_unstale":"2024-04-14T15:10:42.505320+0000","last_undegraded":"2024-04-14T15:10:42.505320+0000","last_fullsized":"2024-04-14T15:10:42.505320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:46:16.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426806+0000","last_change":"2024-04-14T15:10:07.327189+0000","last_active":"2024-04-14T15:10:42.426806+0000","last_peered":"2024-04-14T15:10:42.426806+0000","last_clean":"2024-04-14T15:10:42.426806+0000","last_became_active":"2024-04-14T15:10:07.326830+0000","last_became_peered":"2024-04-14T15:10:07.326830+0000","last_unstale":"2024-04-14T15:10:42.426806+0000","last_undegraded":"2024-04-14T15:10:42.426806+0000","last_fullsized":"2024-04-14T15:10:42.426806+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:44:02.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344606+0000","last_change":"2024-04-14T15:10:05.312335+0000","last_active":"2024-04-14T15:11:16.344606+0000","last_peered":"2024-04-14T15:11:16.344606+0000","last_clean":"2024-04-14T15:11:16.344606+0000","last_became_active":"2024-04-14T15:10:05.312031+0000","last_became_peered":"2024-04-14T15:10:05.312031+0000","last_unstale":"2024-04-14T15:11:16.344606+0000","last_undegraded":"2024-04-14T15:11:16.344606+0000","last_fullsized":"2024-04-14T15:11:16.344606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:17:36.552106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804013+0000","last_change":"2024-04-14T15:10:01.284063+0000","last_active":"2024-04-14T15:10:43.804013+0000","last_peered":"2024-04-14T15:10:43.804013+0000","last_clean":"2024-04-14T15:10:43.804013+0000","last_became_active":"2024-04-14T15:10:01.283759+0000","last_became_peered":"2024-04-14T15:10:01.283759+0000","last_unstale":"2024-04-14T15:10:43.804013+0000","last_undegraded":"2024-04-14T15:10:43.804013+0000","last_fullsized":"2024-04-14T15:10:43.804013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:51:00.638086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.337127+0000","last_change":"2024-04-14T15:10:03.300367+0000","last_active":"2024-04-14T15:10:42.337127+0000","last_peered":"2024-04-14T15:10:42.337127+0000","last_clean":"2024-04-14T15:10:42.337127+0000","last_became_active":"2024-04-14T15:10:03.299175+0000","last_became_peered":"2024-04-14T15:10:03.299175+0000","last_unstale":"2024-04-14T15:10:42.337127+0000","last_undegraded":"2024-04-14T15:10:42.337127+0000","last_fullsized":"2024-04-14T15:10:42.337127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:53:23.786101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804669+0000","last_change":"2024-04-14T15:10:07.637118+0000","last_active":"2024-04-14T15:10:43.804669+0000","last_peered":"2024-04-14T15:10:43.804669+0000","last_clean":"2024-04-14T15:10:43.804669+0000","last_became_active":"2024-04-14T15:10:07.636721+0000","last_became_peered":"2024-04-14T15:10:07.636721+0000","last_unstale":"2024-04-14T15:10:43.804669+0000","last_undegraded":"2024-04-14T15:10:43.804669+0000","last_fullsized":"2024-04-14T15:10:43.804669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:30:07.542573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427383+0000","last_change":"2024-04-14T15:10:05.316726+0000","last_active":"2024-04-14T15:10:42.427383+0000","last_peered":"2024-04-14T15:10:42.427383+0000","last_clean":"2024-04-14T15:10:42.427383+0000","last_became_active":"2024-04-14T15:10:05.315634+0000","last_became_peered":"2024-04-14T15:10:05.315634+0000","last_unstale":"2024-04-14T15:10:42.427383+0000","last_undegraded":"2024-04-14T15:10:42.427383+0000","last_fullsized":"2024-04-14T15:10:42.427383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:29:39.819859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505347+0000","last_change":"2024-04-14T15:10:01.289850+0000","last_active":"2024-04-14T15:10:42.505347+0000","last_peered":"2024-04-14T15:10:42.505347+0000","last_clean":"2024-04-14T15:10:42.505347+0000","last_became_active":"2024-04-14T15:10:01.289516+0000","last_became_peered":"2024-04-14T15:10:01.289516+0000","last_unstale":"2024-04-14T15:10:42.505347+0000","last_undegraded":"2024-04-14T15:10:42.505347+0000","last_fullsized":"2024-04-14T15:10:42.505347+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:31:39.917053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505979+0000","last_change":"2024-04-14T15:10:03.306367+0000","last_active":"2024-04-14T15:10:42.505979+0000","last_peered":"2024-04-14T15:10:42.505979+0000","last_clean":"2024-04-14T15:10:42.505979+0000","last_became_active":"2024-04-14T15:10:03.306075+0000","last_became_peered":"2024-04-14T15:10:03.306075+0000","last_unstale":"2024-04-14T15:10:42.505979+0000","last_undegraded":"2024-04-14T15:10:42.505979+0000","last_fullsized":"2024-04-14T15:10:42.505979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:59:31.179931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339719+0000","last_change":"2024-04-14T15:10:07.316092+0000","last_active":"2024-04-14T15:10:42.339719+0000","last_peered":"2024-04-14T15:10:42.339719+0000","last_clean":"2024-04-14T15:10:42.339719+0000","last_became_active":"2024-04-14T15:10:07.315761+0000","last_became_peered":"2024-04-14T15:10:07.315761+0000","last_unstale":"2024-04-14T15:10:42.339719+0000","last_undegraded":"2024-04-14T15:10:42.339719+0000","last_fullsized":"2024-04-14T15:10:42.339719+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:06:31.680274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336342+0000","last_change":"2024-04-14T15:10:05.310948+0000","last_active":"2024-04-14T15:10:42.336342+0000","last_peered":"2024-04-14T15:10:42.336342+0000","last_clean":"2024-04-14T15:10:42.336342+0000","last_became_active":"2024-04-14T15:10:05.310544+0000","last_became_peered":"2024-04-14T15:10:05.310544+0000","last_unstale":"2024-04-14T15:10:42.336342+0000","last_undegraded":"2024-04-14T15:10:42.336342+0000","last_fullsized":"2024-04-14T15:10:42.336342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:13:46.271225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802982+0000","last_change":"2024-04-14T15:10:01.284704+0000","last_active":"2024-04-14T15:10:43.802982+0000","last_peered":"2024-04-14T15:10:43.802982+0000","last_clean":"2024-04-14T15:10:43.802982+0000","last_became_active":"2024-04-14T15:10:01.284069+0000","last_became_peered":"2024-04-14T15:10:01.284069+0000","last_unstale":"2024-04-14T15:10:43.802982+0000","last_undegraded":"2024-04-14T15:10:43.802982+0000","last_fullsized":"2024-04-14T15:10:43.802982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:27:02.834976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":53,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506065+0000","last_change":"2024-04-14T15:10:03.305992+0000","last_active":"2024-04-14T15:10:42.506065+0000","last_peered":"2024-04-14T15:10:42.506065+0000","last_clean":"2024-04-14T15:10:42.506065+0000","last_became_active":"2024-04-14T15:10:03.305666+0000","last_became_peered":"2024-04-14T15:10:03.305666+0000","last_unstale":"2024-04-14T15:10:42.506065+0000","last_undegraded":"2024-04-14T15:10:42.506065+0000","last_fullsized":"2024-04-14T15:10:42.506065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:22:48.366305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506119+0000","last_change":"2024-04-14T15:10:07.329171+0000","last_active":"2024-04-14T15:10:42.506119+0000","last_peered":"2024-04-14T15:10:42.506119+0000","last_clean":"2024-04-14T15:10:42.506119+0000","last_became_active":"2024-04-14T15:10:07.328890+0000","last_became_peered":"2024-04-14T15:10:07.328890+0000","last_unstale":"2024-04-14T15:10:42.506119+0000","last_undegraded":"2024-04-14T15:10:42.506119+0000","last_fullsized":"2024-04-14T15:10:42.506119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:56:32.325594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344518+0000","last_change":"2024-04-14T15:10:05.313473+0000","last_active":"2024-04-14T15:11:16.344518+0000","last_peered":"2024-04-14T15:11:16.344518+0000","last_clean":"2024-04-14T15:11:16.344518+0000","last_became_active":"2024-04-14T15:10:05.313006+0000","last_became_peered":"2024-04-14T15:10:05.313006+0000","last_unstale":"2024-04-14T15:11:16.344518+0000","last_undegraded":"2024-04-14T15:11:16.344518+0000","last_fullsized":"2024-04-14T15:11:16.344518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:09:23.666868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804682+0000","last_change":"2024-04-14T15:10:01.288658+0000","last_active":"2024-04-14T15:10:43.804682+0000","last_peered":"2024-04-14T15:10:43.804682+0000","last_clean":"2024-04-14T15:10:43.804682+0000","last_became_active":"2024-04-14T15:10:01.288352+0000","last_became_peered":"2024-04-14T15:10:01.288352+0000","last_unstale":"2024-04-14T15:10:43.804682+0000","last_undegraded":"2024-04-14T15:10:43.804682+0000","last_fullsized":"2024-04-14T15:10:43.804682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:11:10.332314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505733+0000","last_change":"2024-04-14T15:10:03.302665+0000","last_active":"2024-04-14T15:10:42.505733+0000","last_peered":"2024-04-14T15:10:42.505733+0000","last_clean":"2024-04-14T15:10:42.505733+0000","last_became_active":"2024-04-14T15:10:03.302422+0000","last_became_peered":"2024-04-14T15:10:03.302422+0000","last_unstale":"2024-04-14T15:10:42.505733+0000","last_undegraded":"2024-04-14T15:10:42.505733+0000","last_fullsized":"2024-04-14T15:10:42.505733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:33:51.914041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712720,"statfs":{"total":95995035648,"available":95961825280,"internally_reserved":0,"allocated":1282048,"data_stored":857385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716892,"statfs":{"total":95995035648,"available":95966097408,"internally_reserved":0,"allocated":1269760,"data_stored":856782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31900,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713252,"statfs":{"total":95995035648,"available":95962370048,"internally_reserved":0,"allocated":737280,"data_stored":268661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28276,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716876,"statfs":{"total":95995035648,"available":95966081024,"internally_reserved":0,"allocated":1286144,"data_stored":857127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717480,"statfs":{"total":95995035648,"available":95966699520,"internally_reserved":0,"allocated":667648,"data_stored":266516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":268216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713288,"statfs":{"total":95995035648,"available":95962406912,"internally_reserved":0,"allocated":700416,"data_stored":267481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717472,"statfs":{"total":95995035648,"available":95966691328,"internally_reserved":0,"allocated":675840,"data_stored":266304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T15:11:25.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph pg dump --format=json 2024-04-14T15:11:26.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:26 smithi045 ceph-mon[29624]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:26.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:26 smithi045 ceph-mon[29624]: from='client.24652 v1:172.21.15.45:0/475397246' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:26 smithi071 ceph-mon[33150]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:26 smithi071 ceph-mon[33150]: from='client.24652 v1:172.21.15.45:0/475397246' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:26.466 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:26 smithi045 ceph-mon[37795]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:26.466 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:26 smithi045 ceph-mon[37795]: from='client.24652 v1:172.21.15.45:0/475397246' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:11:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- 172.21.15.45:0/1671469156 >> v1:172.21.15.45:6789/0 conn(0x7ff90010f330 legacy=0x7ff9001117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- 172.21.15.45:0/1671469156 shutdown_connections 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- 172.21.15.45:0/1671469156 >> 172.21.15.45:0/1671469156 conn(0x7ff9000fe3c0 msgr2=0x7ff9001007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- 172.21.15.45:0/1671469156 shutdown_connections 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- 172.21.15.45:0/1671469156 wait complete. 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 Processor -- start 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.661+0000 7ff9052c5700 1 -- start start 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff9052c5700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff9001a7960 con 0x7ff90010f330 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff9052c5700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff9001a8b80 con 0x7ff9001088b0 2024-04-14T15:11:27.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff9052c5700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7ff9001a9dc0 con 0x7ff90010b760 2024-04-14T15:11:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8f7fff700 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7ff90010b760 0x7ff9001a27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:35104/0 (socket says 172.21.15.45:35104) 2024-04-14T15:11:27.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8f7fff700 1 -- 172.21.15.45:0/3012565086 learned_addr learned my addr 172.21.15.45:0/3012565086 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638626147 0 0) 0x7ff9001a9dc0 con 0x7ff90010b760 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8ec0035b0 con 0x7ff90010b760 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031236656 0 0) 0x7ff9001a7960 con 0x7ff90010f330 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9001a9dc0 con 0x7ff90010f330 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275938032 0 0) 0x7ff9001a8b80 con 0x7ff9001088b0 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9001a7960 con 0x7ff9001088b0 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4235516177 0 0) 0x7ff8ec0035b0 con 0x7ff90010b760 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff9001a8b80 con 0x7ff90010b760 2024-04-14T15:11:27.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff8e8001930 con 0x7ff90010b760 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3546221210 0 0) 0x7ff9001a9dc0 con 0x7ff90010f330 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8ec0035b0 con 0x7ff90010f330 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1004519853 0 0) 0x7ff9001a7960 con 0x7ff9001088b0 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff9001a9dc0 con 0x7ff9001088b0 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff8f8001bc0 con 0x7ff90010f330 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.662+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff8f0000fe0 con 0x7ff9001088b0 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 704657636 0 0) 0x7ff9001a8b80 con 0x7ff90010b760 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 >> v1:172.21.15.71:6789/0 conn(0x7ff9001088b0 legacy=0x7ff90010ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 >> v1:172.21.15.45:6789/0 conn(0x7ff90010f330 legacy=0x7ff9001a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9001ab000 con 0x7ff90010b760 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff9001a7b90 con 0x7ff90010b760 2024-04-14T15:11:27.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff9001a8050 con 0x7ff90010b760 2024-04-14T15:11:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7ff8e8003b30 con 0x7ff90010b760 2024-04-14T15:11:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.663+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7ff8e8005ca0 con 0x7ff90010b760 2024-04-14T15:11:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.664+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e0005150 con 0x7ff90010b760 2024-04-14T15:11:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.664+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7ff8e801c850 con 0x7ff90010b760 2024-04-14T15:11:27.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.665+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7ff8e808a430 con 0x7ff90010b760 2024-04-14T15:11:27.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.669+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7ff8e8057610 con 0x7ff90010b760 2024-04-14T15:11:27.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.871+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 --> v1:172.21.15.45:6800/1718216366 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8e0000bc0 con 0x7ff8ec06fa00 2024-04-14T15:11:27.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.881+0000 7ff8fcff9700 1 -- 172.21.15.45:0/3012565086 <== mgr.14610 v1:172.21.15.45:6800/1718216366 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+346474 (unknown 2965378022 0 2130708546) 0x7ff8e0000bc0 con 0x7ff8ec06fa00 2024-04-14T15:11:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 >> v1:172.21.15.45:6800/1718216366 conn(0x7ff8ec06fa00 legacy=0x7ff8ec071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 >> v1:172.21.15.45:6790/0 conn(0x7ff90010b760 legacy=0x7ff9001a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 shutdown_connections 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 >> 172.21.15.45:0/3012565086 conn(0x7ff9000fe3c0 msgr2=0x7ff900111fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 shutdown_connections 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:27.884+0000 7ff9052c5700 1 -- 172.21.15.45:0/3012565086 wait complete. 2024-04-14T15:11:27.887 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-14T15:11:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:28 smithi071 ceph-mon[33150]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[37795]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[29624]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:28.367 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:28.559 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-14T15:11:26.354807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1010,"num_read_kb":856,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":235804,"kb_used_data":7164,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725412,"statfs":{"total":767960285184,"available":767718821888,"internally_reserved":0,"allocated":7335936,"data_stored":3908472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":234016306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003025"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505382+0000","last_change":"2024-04-14T15:10:07.328416+0000","last_active":"2024-04-14T15:10:42.505382+0000","last_peered":"2024-04-14T15:10:42.505382+0000","last_clean":"2024-04-14T15:10:42.505382+0000","last_became_active":"2024-04-14T15:10:07.328053+0000","last_became_peered":"2024-04-14T15:10:07.328053+0000","last_unstale":"2024-04-14T15:10:42.505382+0000","last_undegraded":"2024-04-14T15:10:42.505382+0000","last_fullsized":"2024-04-14T15:10:42.505382+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:35:00.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335717+0000","last_change":"2024-04-14T15:10:05.316678+0000","last_active":"2024-04-14T15:10:42.335717+0000","last_peered":"2024-04-14T15:10:42.335717+0000","last_clean":"2024-04-14T15:10:42.335717+0000","last_became_active":"2024-04-14T15:10:05.314691+0000","last_became_peered":"2024-04-14T15:10:05.314691+0000","last_unstale":"2024-04-14T15:10:42.335717+0000","last_undegraded":"2024-04-14T15:10:42.335717+0000","last_fullsized":"2024-04-14T15:10:42.335717+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:23:43.814674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803653+0000","last_change":"2024-04-14T15:10:01.290234+0000","last_active":"2024-04-14T15:10:43.803653+0000","last_peered":"2024-04-14T15:10:43.803653+0000","last_clean":"2024-04-14T15:10:43.803653+0000","last_became_active":"2024-04-14T15:10:01.289946+0000","last_became_peered":"2024-04-14T15:10:01.289946+0000","last_unstale":"2024-04-14T15:10:43.803653+0000","last_undegraded":"2024-04-14T15:10:43.803653+0000","last_fullsized":"2024-04-14T15:10:43.803653+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:38:09.696997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505252+0000","last_change":"2024-04-14T15:10:03.306513+0000","last_active":"2024-04-14T15:10:42.505252+0000","last_peered":"2024-04-14T15:10:42.505252+0000","last_clean":"2024-04-14T15:10:42.505252+0000","last_became_active":"2024-04-14T15:10:03.306066+0000","last_became_peered":"2024-04-14T15:10:03.306066+0000","last_unstale":"2024-04-14T15:10:42.505252+0000","last_undegraded":"2024-04-14T15:10:42.505252+0000","last_fullsized":"2024-04-14T15:10:42.505252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:01:46.636333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804789+0000","last_change":"2024-04-14T15:10:07.639197+0000","last_active":"2024-04-14T15:10:43.804789+0000","last_peered":"2024-04-14T15:10:43.804789+0000","last_clean":"2024-04-14T15:10:43.804789+0000","last_became_active":"2024-04-14T15:10:07.638812+0000","last_became_peered":"2024-04-14T15:10:07.638812+0000","last_unstale":"2024-04-14T15:10:43.804789+0000","last_undegraded":"2024-04-14T15:10:43.804789+0000","last_fullsized":"2024-04-14T15:10:43.804789+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:14:16.938555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505273+0000","last_change":"2024-04-14T15:10:05.313030+0000","last_active":"2024-04-14T15:10:42.505273+0000","last_peered":"2024-04-14T15:10:42.505273+0000","last_clean":"2024-04-14T15:10:42.505273+0000","last_became_active":"2024-04-14T15:10:05.312720+0000","last_became_peered":"2024-04-14T15:10:05.312720+0000","last_unstale":"2024-04-14T15:10:42.505273+0000","last_undegraded":"2024-04-14T15:10:42.505273+0000","last_fullsized":"2024-04-14T15:10:42.505273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:10:14.331280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"61'1","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804841+0000","last_change":"2024-04-14T15:10:01.287434+0000","last_active":"2024-04-14T15:10:43.804841+0000","last_peered":"2024-04-14T15:10:43.804841+0000","last_clean":"2024-04-14T15:10:43.804841+0000","last_became_active":"2024-04-14T15:10:01.287134+0000","last_became_peered":"2024-04-14T15:10:01.287134+0000","last_unstale":"2024-04-14T15:10:43.804841+0000","last_undegraded":"2024-04-14T15:10:43.804841+0000","last_fullsized":"2024-04-14T15:10:43.804841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:14:59.013867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803115+0000","last_change":"2024-04-14T15:10:03.298079+0000","last_active":"2024-04-14T15:10:43.803115+0000","last_peered":"2024-04-14T15:10:43.803115+0000","last_clean":"2024-04-14T15:10:43.803115+0000","last_became_active":"2024-04-14T15:10:03.297788+0000","last_became_peered":"2024-04-14T15:10:03.297788+0000","last_unstale":"2024-04-14T15:10:43.803115+0000","last_undegraded":"2024-04-14T15:10:43.803115+0000","last_fullsized":"2024-04-14T15:10:43.803115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:24:54.677004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804940+0000","last_change":"2024-04-14T15:10:07.327110+0000","last_active":"2024-04-14T15:10:43.804940+0000","last_peered":"2024-04-14T15:10:43.804940+0000","last_clean":"2024-04-14T15:10:43.804940+0000","last_became_active":"2024-04-14T15:10:07.326743+0000","last_became_peered":"2024-04-14T15:10:07.326743+0000","last_unstale":"2024-04-14T15:10:43.804940+0000","last_undegraded":"2024-04-14T15:10:43.804940+0000","last_fullsized":"2024-04-14T15:10:43.804940+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:27:40.355052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426656+0000","last_change":"2024-04-14T15:10:05.314296+0000","last_active":"2024-04-14T15:10:42.426656+0000","last_peered":"2024-04-14T15:10:42.426656+0000","last_clean":"2024-04-14T15:10:42.426656+0000","last_became_active":"2024-04-14T15:10:05.313975+0000","last_became_peered":"2024-04-14T15:10:05.313975+0000","last_unstale":"2024-04-14T15:10:42.426656+0000","last_undegraded":"2024-04-14T15:10:42.426656+0000","last_fullsized":"2024-04-14T15:10:42.426656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:10:16.662567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426910+0000","last_change":"2024-04-14T15:10:01.288846+0000","last_active":"2024-04-14T15:10:42.426910+0000","last_peered":"2024-04-14T15:10:42.426910+0000","last_clean":"2024-04-14T15:10:42.426910+0000","last_became_active":"2024-04-14T15:10:01.288167+0000","last_became_peered":"2024-04-14T15:10:01.288167+0000","last_unstale":"2024-04-14T15:10:42.426910+0000","last_undegraded":"2024-04-14T15:10:42.426910+0000","last_fullsized":"2024-04-14T15:10:42.426910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:45:31.589618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505513+0000","last_change":"2024-04-14T15:10:03.303605+0000","last_active":"2024-04-14T15:10:42.505513+0000","last_peered":"2024-04-14T15:10:42.505513+0000","last_clean":"2024-04-14T15:10:42.505513+0000","last_became_active":"2024-04-14T15:10:03.303326+0000","last_became_peered":"2024-04-14T15:10:03.303326+0000","last_unstale":"2024-04-14T15:10:42.505513+0000","last_undegraded":"2024-04-14T15:10:42.505513+0000","last_fullsized":"2024-04-14T15:10:42.505513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:34:53.330974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426439+0000","last_change":"2024-04-14T15:10:07.329923+0000","last_active":"2024-04-14T15:10:42.426439+0000","last_peered":"2024-04-14T15:10:42.426439+0000","last_clean":"2024-04-14T15:10:42.426439+0000","last_became_active":"2024-04-14T15:10:07.329683+0000","last_became_peered":"2024-04-14T15:10:07.329683+0000","last_unstale":"2024-04-14T15:10:42.426439+0000","last_undegraded":"2024-04-14T15:10:42.426439+0000","last_fullsized":"2024-04-14T15:10:42.426439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:56:09.827210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338775+0000","last_change":"2024-04-14T15:10:05.318174+0000","last_active":"2024-04-14T15:10:42.338775+0000","last_peered":"2024-04-14T15:10:42.338775+0000","last_clean":"2024-04-14T15:10:42.338775+0000","last_became_active":"2024-04-14T15:10:05.317854+0000","last_became_peered":"2024-04-14T15:10:05.317854+0000","last_unstale":"2024-04-14T15:10:42.338775+0000","last_undegraded":"2024-04-14T15:10:42.338775+0000","last_fullsized":"2024-04-14T15:10:42.338775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:42:20.184563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426739+0000","last_change":"2024-04-14T15:10:01.294575+0000","last_active":"2024-04-14T15:10:42.426739+0000","last_peered":"2024-04-14T15:10:42.426739+0000","last_clean":"2024-04-14T15:10:42.426739+0000","last_became_active":"2024-04-14T15:10:01.294264+0000","last_became_peered":"2024-04-14T15:10:01.294264+0000","last_unstale":"2024-04-14T15:10:42.426739+0000","last_undegraded":"2024-04-14T15:10:42.426739+0000","last_fullsized":"2024-04-14T15:10:42.426739+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:54:41.706292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":35,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505466+0000","last_change":"2024-04-14T15:10:03.306090+0000","last_active":"2024-04-14T15:10:42.505466+0000","last_peered":"2024-04-14T15:10:42.505466+0000","last_clean":"2024-04-14T15:10:42.505466+0000","last_became_active":"2024-04-14T15:10:03.305580+0000","last_became_peered":"2024-04-14T15:10:03.305580+0000","last_unstale":"2024-04-14T15:10:42.505466+0000","last_undegraded":"2024-04-14T15:10:42.505466+0000","last_fullsized":"2024-04-14T15:10:42.505466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:15:37.116468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506009+0000","last_change":"2024-04-14T15:10:07.636906+0000","last_active":"2024-04-14T15:10:42.506009+0000","last_peered":"2024-04-14T15:10:42.506009+0000","last_clean":"2024-04-14T15:10:42.506009+0000","last_became_active":"2024-04-14T15:10:07.636559+0000","last_became_peered":"2024-04-14T15:10:07.636559+0000","last_unstale":"2024-04-14T15:10:42.506009+0000","last_undegraded":"2024-04-14T15:10:42.506009+0000","last_fullsized":"2024-04-14T15:10:42.506009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:58:13.665827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336625+0000","last_change":"2024-04-14T15:10:05.310958+0000","last_active":"2024-04-14T15:10:42.336625+0000","last_peered":"2024-04-14T15:10:42.336625+0000","last_clean":"2024-04-14T15:10:42.336625+0000","last_became_active":"2024-04-14T15:10:05.310586+0000","last_became_peered":"2024-04-14T15:10:05.310586+0000","last_unstale":"2024-04-14T15:10:42.336625+0000","last_undegraded":"2024-04-14T15:10:42.336625+0000","last_fullsized":"2024-04-14T15:10:42.336625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:54:04.107918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.806175+0000","last_change":"2024-04-14T15:10:01.292218+0000","last_active":"2024-04-14T15:10:43.806175+0000","last_peered":"2024-04-14T15:10:43.806175+0000","last_clean":"2024-04-14T15:10:43.806175+0000","last_became_active":"2024-04-14T15:10:01.291952+0000","last_became_peered":"2024-04-14T15:10:01.291952+0000","last_unstale":"2024-04-14T15:10:43.806175+0000","last_undegraded":"2024-04-14T15:10:43.806175+0000","last_fullsized":"2024-04-14T15:10:43.806175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:30:05.912982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339014+0000","last_change":"2024-04-14T15:10:03.298476+0000","last_active":"2024-04-14T15:10:42.339014+0000","last_peered":"2024-04-14T15:10:42.339014+0000","last_clean":"2024-04-14T15:10:42.339014+0000","last_became_active":"2024-04-14T15:10:03.298122+0000","last_became_peered":"2024-04-14T15:10:03.298122+0000","last_unstale":"2024-04-14T15:10:42.339014+0000","last_undegraded":"2024-04-14T15:10:42.339014+0000","last_fullsized":"2024-04-14T15:10:42.339014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:22:18.725525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805385+0000","last_change":"2024-04-14T15:10:07.637163+0000","last_active":"2024-04-14T15:10:43.805385+0000","last_peered":"2024-04-14T15:10:43.805385+0000","last_clean":"2024-04-14T15:10:43.805385+0000","last_became_active":"2024-04-14T15:10:07.636777+0000","last_became_peered":"2024-04-14T15:10:07.636777+0000","last_unstale":"2024-04-14T15:10:43.805385+0000","last_undegraded":"2024-04-14T15:10:43.805385+0000","last_fullsized":"2024-04-14T15:10:43.805385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:24:24.611904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505623+0000","last_change":"2024-04-14T15:10:05.316956+0000","last_active":"2024-04-14T15:10:42.505623+0000","last_peered":"2024-04-14T15:10:42.505623+0000","last_clean":"2024-04-14T15:10:42.505623+0000","last_became_active":"2024-04-14T15:10:05.315776+0000","last_became_peered":"2024-04-14T15:10:05.315776+0000","last_unstale":"2024-04-14T15:10:42.505623+0000","last_undegraded":"2024-04-14T15:10:42.505623+0000","last_fullsized":"2024-04-14T15:10:42.505623+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:29:19.558266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505644+0000","last_change":"2024-04-14T15:10:01.277242+0000","last_active":"2024-04-14T15:10:42.505644+0000","last_peered":"2024-04-14T15:10:42.505644+0000","last_clean":"2024-04-14T15:10:42.505644+0000","last_became_active":"2024-04-14T15:10:01.276939+0000","last_became_peered":"2024-04-14T15:10:01.276939+0000","last_unstale":"2024-04-14T15:10:42.505644+0000","last_undegraded":"2024-04-14T15:10:42.505644+0000","last_fullsized":"2024-04-14T15:10:42.505644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:35:53.662204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803418+0000","last_change":"2024-04-14T15:10:03.300602+0000","last_active":"2024-04-14T15:10:43.803418+0000","last_peered":"2024-04-14T15:10:43.803418+0000","last_clean":"2024-04-14T15:10:43.803418+0000","last_became_active":"2024-04-14T15:10:03.300270+0000","last_became_peered":"2024-04-14T15:10:03.300270+0000","last_unstale":"2024-04-14T15:10:43.803418+0000","last_undegraded":"2024-04-14T15:10:43.803418+0000","last_fullsized":"2024-04-14T15:10:43.803418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:27:34.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338972+0000","last_change":"2024-04-14T15:10:07.327405+0000","last_active":"2024-04-14T15:10:42.338972+0000","last_peered":"2024-04-14T15:10:42.338972+0000","last_clean":"2024-04-14T15:10:42.338972+0000","last_became_active":"2024-04-14T15:10:07.326994+0000","last_became_peered":"2024-04-14T15:10:07.326994+0000","last_unstale":"2024-04-14T15:10:42.338972+0000","last_undegraded":"2024-04-14T15:10:42.338972+0000","last_fullsized":"2024-04-14T15:10:42.338972+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:09:00.228979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344606+0000","last_change":"2024-04-14T15:10:05.317523+0000","last_active":"2024-04-14T15:11:16.344606+0000","last_peered":"2024-04-14T15:11:16.344606+0000","last_clean":"2024-04-14T15:11:16.344606+0000","last_became_active":"2024-04-14T15:10:05.317188+0000","last_became_peered":"2024-04-14T15:10:05.317188+0000","last_unstale":"2024-04-14T15:11:16.344606+0000","last_undegraded":"2024-04-14T15:11:16.344606+0000","last_fullsized":"2024-04-14T15:11:16.344606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:45:07.548659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'2","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335833+0000","last_change":"2024-04-14T15:10:01.280775+0000","last_active":"2024-04-14T15:10:42.335833+0000","last_peered":"2024-04-14T15:10:42.335833+0000","last_clean":"2024-04-14T15:10:42.335833+0000","last_became_active":"2024-04-14T15:10:01.280448+0000","last_became_peered":"2024-04-14T15:10:01.280448+0000","last_unstale":"2024-04-14T15:10:42.335833+0000","last_undegraded":"2024-04-14T15:10:42.335833+0000","last_fullsized":"2024-04-14T15:10:42.335833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:30:21.765182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":957,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805737+0000","last_change":"2024-04-14T15:10:03.966220+0000","last_active":"2024-04-14T15:10:43.805737+0000","last_peered":"2024-04-14T15:10:43.805737+0000","last_clean":"2024-04-14T15:10:43.805737+0000","last_became_active":"2024-04-14T15:10:03.965030+0000","last_became_peered":"2024-04-14T15:10:03.965030+0000","last_unstale":"2024-04-14T15:10:43.805737+0000","last_undegraded":"2024-04-14T15:10:43.805737+0000","last_fullsized":"2024-04-14T15:10:43.805737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:15:43.561734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336277+0000","last_change":"2024-04-14T15:10:07.326847+0000","last_active":"2024-04-14T15:10:42.336277+0000","last_peered":"2024-04-14T15:10:42.336277+0000","last_clean":"2024-04-14T15:10:42.336277+0000","last_became_active":"2024-04-14T15:10:07.326530+0000","last_became_peered":"2024-04-14T15:10:07.326530+0000","last_unstale":"2024-04-14T15:10:42.336277+0000","last_undegraded":"2024-04-14T15:10:42.336277+0000","last_fullsized":"2024-04-14T15:10:42.336277+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:31:26.084295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.806048+0000","last_change":"2024-04-14T15:10:05.314266+0000","last_active":"2024-04-14T15:10:43.806048+0000","last_peered":"2024-04-14T15:10:43.806048+0000","last_clean":"2024-04-14T15:10:43.806048+0000","last_became_active":"2024-04-14T15:10:05.313939+0000","last_became_peered":"2024-04-14T15:10:05.313939+0000","last_unstale":"2024-04-14T15:10:43.806048+0000","last_undegraded":"2024-04-14T15:10:43.806048+0000","last_fullsized":"2024-04-14T15:10:43.806048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:43:23.591510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803897+0000","last_change":"2024-04-14T15:10:01.289257+0000","last_active":"2024-04-14T15:10:43.803897+0000","last_peered":"2024-04-14T15:10:43.803897+0000","last_clean":"2024-04-14T15:10:43.803897+0000","last_became_active":"2024-04-14T15:10:01.288972+0000","last_became_peered":"2024-04-14T15:10:01.288972+0000","last_unstale":"2024-04-14T15:10:43.803897+0000","last_undegraded":"2024-04-14T15:10:43.803897+0000","last_fullsized":"2024-04-14T15:10:43.803897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:02:15.791539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802653+0000","last_change":"2024-04-14T15:10:03.299826+0000","last_active":"2024-04-14T15:10:43.802653+0000","last_peered":"2024-04-14T15:10:43.802653+0000","last_clean":"2024-04-14T15:10:43.802653+0000","last_became_active":"2024-04-14T15:10:03.299481+0000","last_became_peered":"2024-04-14T15:10:03.299481+0000","last_unstale":"2024-04-14T15:10:43.802653+0000","last_undegraded":"2024-04-14T15:10:43.802653+0000","last_fullsized":"2024-04-14T15:10:43.802653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:58:57.232199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338718+0000","last_change":"2024-04-14T15:10:07.327521+0000","last_active":"2024-04-14T15:10:42.338718+0000","last_peered":"2024-04-14T15:10:42.338718+0000","last_clean":"2024-04-14T15:10:42.338718+0000","last_became_active":"2024-04-14T15:10:07.327090+0000","last_became_peered":"2024-04-14T15:10:07.327090+0000","last_unstale":"2024-04-14T15:10:42.338718+0000","last_undegraded":"2024-04-14T15:10:42.338718+0000","last_fullsized":"2024-04-14T15:10:42.338718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:59:51.862824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802847+0000","last_change":"2024-04-14T15:10:05.316779+0000","last_active":"2024-04-14T15:10:43.802847+0000","last_peered":"2024-04-14T15:10:43.802847+0000","last_clean":"2024-04-14T15:10:43.802847+0000","last_became_active":"2024-04-14T15:10:05.316439+0000","last_became_peered":"2024-04-14T15:10:05.316439+0000","last_unstale":"2024-04-14T15:10:43.802847+0000","last_undegraded":"2024-04-14T15:10:43.802847+0000","last_fullsized":"2024-04-14T15:10:43.802847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:36:59.089280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803554+0000","last_change":"2024-04-14T15:10:01.291409+0000","last_active":"2024-04-14T15:10:43.803554+0000","last_peered":"2024-04-14T15:10:43.803554+0000","last_clean":"2024-04-14T15:10:43.803554+0000","last_became_active":"2024-04-14T15:10:01.291121+0000","last_became_peered":"2024-04-14T15:10:01.291121+0000","last_unstale":"2024-04-14T15:10:43.803554+0000","last_undegraded":"2024-04-14T15:10:43.803554+0000","last_fullsized":"2024-04-14T15:10:43.803554+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:37:58.478664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802941+0000","last_change":"2024-04-14T15:10:03.300644+0000","last_active":"2024-04-14T15:10:43.802941+0000","last_peered":"2024-04-14T15:10:43.802941+0000","last_clean":"2024-04-14T15:10:43.802941+0000","last_became_active":"2024-04-14T15:10:03.300369+0000","last_became_peered":"2024-04-14T15:10:03.300369+0000","last_unstale":"2024-04-14T15:10:43.802941+0000","last_undegraded":"2024-04-14T15:10:43.802941+0000","last_fullsized":"2024-04-14T15:10:43.802941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:02:43.981261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804386+0000","last_change":"2024-04-14T15:10:07.639183+0000","last_active":"2024-04-14T15:10:43.804386+0000","last_peered":"2024-04-14T15:10:43.804386+0000","last_clean":"2024-04-14T15:10:43.804386+0000","last_became_active":"2024-04-14T15:10:07.638794+0000","last_became_peered":"2024-04-14T15:10:07.638794+0000","last_unstale":"2024-04-14T15:10:43.804386+0000","last_undegraded":"2024-04-14T15:10:43.804386+0000","last_fullsized":"2024-04-14T15:10:43.804386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:42:43.548702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339150+0000","last_change":"2024-04-14T15:10:05.321002+0000","last_active":"2024-04-14T15:10:42.339150+0000","last_peered":"2024-04-14T15:10:42.339150+0000","last_clean":"2024-04-14T15:10:42.339150+0000","last_became_active":"2024-04-14T15:10:05.320683+0000","last_became_peered":"2024-04-14T15:10:05.320683+0000","last_unstale":"2024-04-14T15:10:42.339150+0000","last_undegraded":"2024-04-14T15:10:42.339150+0000","last_fullsized":"2024-04-14T15:10:42.339150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:39:29.409123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426979+0000","last_change":"2024-04-14T15:10:01.288082+0000","last_active":"2024-04-14T15:10:42.426979+0000","last_peered":"2024-04-14T15:10:42.426979+0000","last_clean":"2024-04-14T15:10:42.426979+0000","last_became_active":"2024-04-14T15:10:01.287735+0000","last_became_peered":"2024-04-14T15:10:01.287735+0000","last_unstale":"2024-04-14T15:10:42.426979+0000","last_undegraded":"2024-04-14T15:10:42.426979+0000","last_fullsized":"2024-04-14T15:10:42.426979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:36:00.593000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804095+0000","last_change":"2024-04-14T15:10:03.965980+0000","last_active":"2024-04-14T15:10:43.804095+0000","last_peered":"2024-04-14T15:10:43.804095+0000","last_clean":"2024-04-14T15:10:43.804095+0000","last_became_active":"2024-04-14T15:10:03.965633+0000","last_became_peered":"2024-04-14T15:10:03.965633+0000","last_unstale":"2024-04-14T15:10:43.804095+0000","last_undegraded":"2024-04-14T15:10:43.804095+0000","last_fullsized":"2024-04-14T15:10:43.804095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:49:32.031950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803783+0000","last_change":"2024-04-14T15:10:07.330577+0000","last_active":"2024-04-14T15:10:43.803783+0000","last_peered":"2024-04-14T15:10:43.803783+0000","last_clean":"2024-04-14T15:10:43.803783+0000","last_became_active":"2024-04-14T15:10:07.330215+0000","last_became_peered":"2024-04-14T15:10:07.330215+0000","last_unstale":"2024-04-14T15:10:43.803783+0000","last_undegraded":"2024-04-14T15:10:43.803783+0000","last_fullsized":"2024-04-14T15:10:43.803783+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:01:18.146878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804593+0000","last_change":"2024-04-14T15:10:05.312931+0000","last_active":"2024-04-14T15:10:43.804593+0000","last_peered":"2024-04-14T15:10:43.804593+0000","last_clean":"2024-04-14T15:10:43.804593+0000","last_became_active":"2024-04-14T15:10:05.312409+0000","last_became_peered":"2024-04-14T15:10:05.312409+0000","last_unstale":"2024-04-14T15:10:43.804593+0000","last_undegraded":"2024-04-14T15:10:43.804593+0000","last_fullsized":"2024-04-14T15:10:43.804593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:20:35.448982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339250+0000","last_change":"2024-04-14T15:10:01.294947+0000","last_active":"2024-04-14T15:10:42.339250+0000","last_peered":"2024-04-14T15:10:42.339250+0000","last_clean":"2024-04-14T15:10:42.339250+0000","last_became_active":"2024-04-14T15:10:01.294623+0000","last_became_peered":"2024-04-14T15:10:01.294623+0000","last_unstale":"2024-04-14T15:10:42.339250+0000","last_undegraded":"2024-04-14T15:10:42.339250+0000","last_fullsized":"2024-04-14T15:10:42.339250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:42:55.865288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335999+0000","last_change":"2024-04-14T15:10:03.302392+0000","last_active":"2024-04-14T15:10:42.335999+0000","last_peered":"2024-04-14T15:10:42.335999+0000","last_clean":"2024-04-14T15:10:42.335999+0000","last_became_active":"2024-04-14T15:10:03.301539+0000","last_became_peered":"2024-04-14T15:10:03.301539+0000","last_unstale":"2024-04-14T15:10:42.335999+0000","last_undegraded":"2024-04-14T15:10:42.335999+0000","last_fullsized":"2024-04-14T15:10:42.335999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:44:46.421460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803848+0000","last_change":"2024-04-14T15:10:07.321767+0000","last_active":"2024-04-14T15:10:43.803848+0000","last_peered":"2024-04-14T15:10:43.803848+0000","last_clean":"2024-04-14T15:10:43.803848+0000","last_became_active":"2024-04-14T15:10:07.321406+0000","last_became_peered":"2024-04-14T15:10:07.321406+0000","last_unstale":"2024-04-14T15:10:43.803848+0000","last_undegraded":"2024-04-14T15:10:43.803848+0000","last_fullsized":"2024-04-14T15:10:43.803848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:09:27.865027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335858+0000","last_change":"2024-04-14T15:10:05.317553+0000","last_active":"2024-04-14T15:10:42.335858+0000","last_peered":"2024-04-14T15:10:42.335858+0000","last_clean":"2024-04-14T15:10:42.335858+0000","last_became_active":"2024-04-14T15:10:05.317254+0000","last_became_peered":"2024-04-14T15:10:05.317254+0000","last_unstale":"2024-04-14T15:10:42.335858+0000","last_undegraded":"2024-04-14T15:10:42.335858+0000","last_fullsized":"2024-04-14T15:10:42.335858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:06:43.203130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505865+0000","last_change":"2024-04-14T15:10:01.292065+0000","last_active":"2024-04-14T15:10:42.505865+0000","last_peered":"2024-04-14T15:10:42.505865+0000","last_clean":"2024-04-14T15:10:42.505865+0000","last_became_active":"2024-04-14T15:10:01.291533+0000","last_became_peered":"2024-04-14T15:10:01.291533+0000","last_unstale":"2024-04-14T15:10:42.505865+0000","last_undegraded":"2024-04-14T15:10:42.505865+0000","last_fullsized":"2024-04-14T15:10:42.505865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:23:17.225362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505702+0000","last_change":"2024-04-14T15:10:03.300755+0000","last_active":"2024-04-14T15:10:42.505702+0000","last_peered":"2024-04-14T15:10:42.505702+0000","last_clean":"2024-04-14T15:10:42.505702+0000","last_became_active":"2024-04-14T15:10:03.299638+0000","last_became_peered":"2024-04-14T15:10:03.299638+0000","last_unstale":"2024-04-14T15:10:42.505702+0000","last_undegraded":"2024-04-14T15:10:42.505702+0000","last_fullsized":"2024-04-14T15:10:42.505702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:02:21.922301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805516+0000","last_change":"2024-04-14T15:10:07.323074+0000","last_active":"2024-04-14T15:10:43.805516+0000","last_peered":"2024-04-14T15:10:43.805516+0000","last_clean":"2024-04-14T15:10:43.805516+0000","last_became_active":"2024-04-14T15:10:07.322755+0000","last_became_peered":"2024-04-14T15:10:07.322755+0000","last_unstale":"2024-04-14T15:10:43.805516+0000","last_undegraded":"2024-04-14T15:10:43.805516+0000","last_fullsized":"2024-04-14T15:10:43.805516+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:33:14.138637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":49,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344605+0000","last_change":"2024-04-14T15:10:05.318044+0000","last_active":"2024-04-14T15:11:16.344605+0000","last_peered":"2024-04-14T15:11:16.344605+0000","last_clean":"2024-04-14T15:11:16.344605+0000","last_became_active":"2024-04-14T15:10:05.317538+0000","last_became_peered":"2024-04-14T15:10:05.317538+0000","last_unstale":"2024-04-14T15:11:16.344605+0000","last_undegraded":"2024-04-14T15:11:16.344605+0000","last_fullsized":"2024-04-14T15:11:16.344605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:40:09.782802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338898+0000","last_change":"2024-04-14T15:10:01.285162+0000","last_active":"2024-04-14T15:10:42.338898+0000","last_peered":"2024-04-14T15:10:42.338898+0000","last_clean":"2024-04-14T15:10:42.338898+0000","last_became_active":"2024-04-14T15:10:01.284870+0000","last_became_peered":"2024-04-14T15:10:01.284870+0000","last_unstale":"2024-04-14T15:10:42.338898+0000","last_undegraded":"2024-04-14T15:10:42.338898+0000","last_fullsized":"2024-04-14T15:10:42.338898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:37:29.313852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805034+0000","last_change":"2024-04-14T15:10:03.965870+0000","last_active":"2024-04-14T15:10:43.805034+0000","last_peered":"2024-04-14T15:10:43.805034+0000","last_clean":"2024-04-14T15:10:43.805034+0000","last_became_active":"2024-04-14T15:10:03.965006+0000","last_became_peered":"2024-04-14T15:10:03.965006+0000","last_unstale":"2024-04-14T15:10:43.805034+0000","last_undegraded":"2024-04-14T15:10:43.805034+0000","last_fullsized":"2024-04-14T15:10:43.805034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:40:50.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506147+0000","last_change":"2024-04-14T15:10:07.322845+0000","last_active":"2024-04-14T15:10:42.506147+0000","last_peered":"2024-04-14T15:10:42.506147+0000","last_clean":"2024-04-14T15:10:42.506147+0000","last_became_active":"2024-04-14T15:10:07.322402+0000","last_became_peered":"2024-04-14T15:10:07.322402+0000","last_unstale":"2024-04-14T15:10:42.506147+0000","last_undegraded":"2024-04-14T15:10:42.506147+0000","last_fullsized":"2024-04-14T15:10:42.506147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:36:57.542575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344476+0000","last_change":"2024-04-14T15:10:05.315733+0000","last_active":"2024-04-14T15:11:16.344476+0000","last_peered":"2024-04-14T15:11:16.344476+0000","last_clean":"2024-04-14T15:11:16.344476+0000","last_became_active":"2024-04-14T15:10:05.315375+0000","last_became_peered":"2024-04-14T15:10:05.315375+0000","last_unstale":"2024-04-14T15:11:16.344476+0000","last_undegraded":"2024-04-14T15:11:16.344476+0000","last_fullsized":"2024-04-14T15:11:16.344476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:35:09.538883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803218+0000","last_change":"2024-04-14T15:10:01.292508+0000","last_active":"2024-04-14T15:10:43.803218+0000","last_peered":"2024-04-14T15:10:43.803218+0000","last_clean":"2024-04-14T15:10:43.803218+0000","last_became_active":"2024-04-14T15:10:01.292048+0000","last_became_peered":"2024-04-14T15:10:01.292048+0000","last_unstale":"2024-04-14T15:10:43.803218+0000","last_undegraded":"2024-04-14T15:10:43.803218+0000","last_fullsized":"2024-04-14T15:10:43.803218+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:36:22.472166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427125+0000","last_change":"2024-04-14T15:10:03.965927+0000","last_active":"2024-04-14T15:10:42.427125+0000","last_peered":"2024-04-14T15:10:42.427125+0000","last_clean":"2024-04-14T15:10:42.427125+0000","last_became_active":"2024-04-14T15:10:03.965553+0000","last_became_peered":"2024-04-14T15:10:03.965553+0000","last_unstale":"2024-04-14T15:10:42.427125+0000","last_undegraded":"2024-04-14T15:10:42.427125+0000","last_fullsized":"2024-04-14T15:10:42.427125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:06:55.504599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339383+0000","last_change":"2024-04-14T15:10:07.321561+0000","last_active":"2024-04-14T15:10:42.339383+0000","last_peered":"2024-04-14T15:10:42.339383+0000","last_clean":"2024-04-14T15:10:42.339383+0000","last_became_active":"2024-04-14T15:10:07.320311+0000","last_became_peered":"2024-04-14T15:10:07.320311+0000","last_unstale":"2024-04-14T15:10:42.339383+0000","last_undegraded":"2024-04-14T15:10:42.339383+0000","last_fullsized":"2024-04-14T15:10:42.339383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:32:09.072546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805516+0000","last_change":"2024-04-14T15:10:05.316174+0000","last_active":"2024-04-14T15:10:43.805516+0000","last_peered":"2024-04-14T15:10:43.805516+0000","last_clean":"2024-04-14T15:10:43.805516+0000","last_became_active":"2024-04-14T15:10:05.315807+0000","last_became_peered":"2024-04-14T15:10:05.315807+0000","last_unstale":"2024-04-14T15:10:43.805516+0000","last_undegraded":"2024-04-14T15:10:43.805516+0000","last_fullsized":"2024-04-14T15:10:43.805516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:43:37.742051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339466+0000","last_change":"2024-04-14T15:10:01.289169+0000","last_active":"2024-04-14T15:10:42.339466+0000","last_peered":"2024-04-14T15:10:42.339466+0000","last_clean":"2024-04-14T15:10:42.339466+0000","last_became_active":"2024-04-14T15:10:01.288924+0000","last_became_peered":"2024-04-14T15:10:01.288924+0000","last_unstale":"2024-04-14T15:10:42.339466+0000","last_undegraded":"2024-04-14T15:10:42.339466+0000","last_fullsized":"2024-04-14T15:10:42.339466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:25:27.657428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805864+0000","last_change":"2024-04-14T15:10:03.966576+0000","last_active":"2024-04-14T15:10:43.805864+0000","last_peered":"2024-04-14T15:10:43.805864+0000","last_clean":"2024-04-14T15:10:43.805864+0000","last_became_active":"2024-04-14T15:10:03.965869+0000","last_became_peered":"2024-04-14T15:10:03.965869+0000","last_unstale":"2024-04-14T15:10:43.805864+0000","last_undegraded":"2024-04-14T15:10:43.805864+0000","last_fullsized":"2024-04-14T15:10:43.805864+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:00:04.336610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339141+0000","last_change":"2024-04-14T15:10:07.639897+0000","last_active":"2024-04-14T15:10:42.339141+0000","last_peered":"2024-04-14T15:10:42.339141+0000","last_clean":"2024-04-14T15:10:42.339141+0000","last_became_active":"2024-04-14T15:10:07.639314+0000","last_became_peered":"2024-04-14T15:10:07.639314+0000","last_unstale":"2024-04-14T15:10:42.339141+0000","last_undegraded":"2024-04-14T15:10:42.339141+0000","last_fullsized":"2024-04-14T15:10:42.339141+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:50:22.004118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426982+0000","last_change":"2024-04-14T15:10:05.317039+0000","last_active":"2024-04-14T15:10:42.426982+0000","last_peered":"2024-04-14T15:10:42.426982+0000","last_clean":"2024-04-14T15:10:42.426982+0000","last_became_active":"2024-04-14T15:10:05.316733+0000","last_became_peered":"2024-04-14T15:10:05.316733+0000","last_unstale":"2024-04-14T15:10:42.426982+0000","last_undegraded":"2024-04-14T15:10:42.426982+0000","last_fullsized":"2024-04-14T15:10:42.426982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:38:55.116276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802736+0000","last_change":"2024-04-14T15:10:01.293838+0000","last_active":"2024-04-14T15:10:43.802736+0000","last_peered":"2024-04-14T15:10:43.802736+0000","last_clean":"2024-04-14T15:10:43.802736+0000","last_became_active":"2024-04-14T15:10:01.293517+0000","last_became_peered":"2024-04-14T15:10:01.293517+0000","last_unstale":"2024-04-14T15:10:43.802736+0000","last_undegraded":"2024-04-14T15:10:43.802736+0000","last_fullsized":"2024-04-14T15:10:43.802736+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:24:27.988036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805559+0000","last_change":"2024-04-14T15:10:03.965531+0000","last_active":"2024-04-14T15:10:43.805559+0000","last_peered":"2024-04-14T15:10:43.805559+0000","last_clean":"2024-04-14T15:10:43.805559+0000","last_became_active":"2024-04-14T15:10:03.965047+0000","last_became_peered":"2024-04-14T15:10:03.965047+0000","last_unstale":"2024-04-14T15:10:43.805559+0000","last_undegraded":"2024-04-14T15:10:43.805559+0000","last_fullsized":"2024-04-14T15:10:43.805559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:32:32.285818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426723+0000","last_change":"2024-04-14T15:10:07.637285+0000","last_active":"2024-04-14T15:10:42.426723+0000","last_peered":"2024-04-14T15:10:42.426723+0000","last_clean":"2024-04-14T15:10:42.426723+0000","last_became_active":"2024-04-14T15:10:07.636996+0000","last_became_peered":"2024-04-14T15:10:07.636996+0000","last_unstale":"2024-04-14T15:10:42.426723+0000","last_undegraded":"2024-04-14T15:10:42.426723+0000","last_fullsized":"2024-04-14T15:10:42.426723+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:30:23.406344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339659+0000","last_change":"2024-04-14T15:10:01.295413+0000","last_active":"2024-04-14T15:10:42.339659+0000","last_peered":"2024-04-14T15:10:42.339659+0000","last_clean":"2024-04-14T15:10:42.339659+0000","last_became_active":"2024-04-14T15:10:01.295169+0000","last_became_peered":"2024-04-14T15:10:01.295169+0000","last_unstale":"2024-04-14T15:10:42.339659+0000","last_undegraded":"2024-04-14T15:10:42.339659+0000","last_fullsized":"2024-04-14T15:10:42.339659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:47:48.534661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426471+0000","last_change":"2024-04-14T15:10:03.965452+0000","last_active":"2024-04-14T15:10:42.426471+0000","last_peered":"2024-04-14T15:10:42.426471+0000","last_clean":"2024-04-14T15:10:42.426471+0000","last_became_active":"2024-04-14T15:10:03.965017+0000","last_became_peered":"2024-04-14T15:10:03.965017+0000","last_unstale":"2024-04-14T15:10:42.426471+0000","last_undegraded":"2024-04-14T15:10:42.426471+0000","last_fullsized":"2024-04-14T15:10:42.426471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:37:18.308142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344612+0000","last_change":"2024-04-14T15:10:05.317320+0000","last_active":"2024-04-14T15:11:16.344612+0000","last_peered":"2024-04-14T15:11:16.344612+0000","last_clean":"2024-04-14T15:11:16.344612+0000","last_became_active":"2024-04-14T15:10:05.317004+0000","last_became_peered":"2024-04-14T15:10:05.317004+0000","last_unstale":"2024-04-14T15:11:16.344612+0000","last_undegraded":"2024-04-14T15:11:16.344612+0000","last_fullsized":"2024-04-14T15:11:16.344612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:49:14.442926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804483+0000","last_change":"2024-04-14T15:10:07.323919+0000","last_active":"2024-04-14T15:10:43.804483+0000","last_peered":"2024-04-14T15:10:43.804483+0000","last_clean":"2024-04-14T15:10:43.804483+0000","last_became_active":"2024-04-14T15:10:07.323558+0000","last_became_peered":"2024-04-14T15:10:07.323558+0000","last_unstale":"2024-04-14T15:10:43.804483+0000","last_undegraded":"2024-04-14T15:10:43.804483+0000","last_fullsized":"2024-04-14T15:10:43.804483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:19:05.570992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339384+0000","last_change":"2024-04-14T15:10:01.286196+0000","last_active":"2024-04-14T15:10:42.339384+0000","last_peered":"2024-04-14T15:10:42.339384+0000","last_clean":"2024-04-14T15:10:42.339384+0000","last_became_active":"2024-04-14T15:10:01.285912+0000","last_became_peered":"2024-04-14T15:10:01.285912+0000","last_unstale":"2024-04-14T15:10:42.339384+0000","last_undegraded":"2024-04-14T15:10:42.339384+0000","last_fullsized":"2024-04-14T15:10:42.339384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:25:42.738433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":48,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505605+0000","last_change":"2024-04-14T15:10:03.299918+0000","last_active":"2024-04-14T15:10:42.505605+0000","last_peered":"2024-04-14T15:10:42.505605+0000","last_clean":"2024-04-14T15:10:42.505605+0000","last_became_active":"2024-04-14T15:10:03.299558+0000","last_became_peered":"2024-04-14T15:10:03.299558+0000","last_unstale":"2024-04-14T15:10:42.505605+0000","last_undegraded":"2024-04-14T15:10:42.505605+0000","last_fullsized":"2024-04-14T15:10:42.505605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:08:59.005676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338377+0000","last_change":"2024-04-14T15:10:05.309890+0000","last_active":"2024-04-14T15:10:42.338377+0000","last_peered":"2024-04-14T15:10:42.338377+0000","last_clean":"2024-04-14T15:10:42.338377+0000","last_became_active":"2024-04-14T15:10:05.309549+0000","last_became_peered":"2024-04-14T15:10:05.309549+0000","last_unstale":"2024-04-14T15:10:42.338377+0000","last_undegraded":"2024-04-14T15:10:42.338377+0000","last_fullsized":"2024-04-14T15:10:42.338377+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:33:49.020700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338599+0000","last_change":"2024-04-14T15:10:07.321998+0000","last_active":"2024-04-14T15:10:42.338599+0000","last_peered":"2024-04-14T15:10:42.338599+0000","last_clean":"2024-04-14T15:10:42.338599+0000","last_became_active":"2024-04-14T15:10:07.321680+0000","last_became_peered":"2024-04-14T15:10:07.321680+0000","last_unstale":"2024-04-14T15:10:42.338599+0000","last_undegraded":"2024-04-14T15:10:42.338599+0000","last_fullsized":"2024-04-14T15:10:42.338599+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:37:08.760904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339122+0000","last_change":"2024-04-14T15:10:05.314886+0000","last_active":"2024-04-14T15:10:42.339122+0000","last_peered":"2024-04-14T15:10:42.339122+0000","last_clean":"2024-04-14T15:10:42.339122+0000","last_became_active":"2024-04-14T15:10:05.314443+0000","last_became_peered":"2024-04-14T15:10:05.314443+0000","last_unstale":"2024-04-14T15:10:42.339122+0000","last_undegraded":"2024-04-14T15:10:42.339122+0000","last_fullsized":"2024-04-14T15:10:42.339122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:14:28.738736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339350+0000","last_change":"2024-04-14T15:10:01.295011+0000","last_active":"2024-04-14T15:10:42.339350+0000","last_peered":"2024-04-14T15:10:42.339350+0000","last_clean":"2024-04-14T15:10:42.339350+0000","last_became_active":"2024-04-14T15:10:01.294717+0000","last_became_peered":"2024-04-14T15:10:01.294717+0000","last_unstale":"2024-04-14T15:10:42.339350+0000","last_undegraded":"2024-04-14T15:10:42.339350+0000","last_fullsized":"2024-04-14T15:10:42.339350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:29:40.393469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":56,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803196+0000","last_change":"2024-04-14T15:10:03.965210+0000","last_active":"2024-04-14T15:10:43.803196+0000","last_peered":"2024-04-14T15:10:43.803196+0000","last_clean":"2024-04-14T15:10:43.803196+0000","last_became_active":"2024-04-14T15:10:03.964870+0000","last_became_peered":"2024-04-14T15:10:03.964870+0000","last_unstale":"2024-04-14T15:10:43.803196+0000","last_undegraded":"2024-04-14T15:10:43.803196+0000","last_fullsized":"2024-04-14T15:10:43.803196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:01:52.144341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803967+0000","last_change":"2024-04-14T15:10:07.326689+0000","last_active":"2024-04-14T15:10:43.803967+0000","last_peered":"2024-04-14T15:10:43.803967+0000","last_clean":"2024-04-14T15:10:43.803967+0000","last_became_active":"2024-04-14T15:10:07.326206+0000","last_became_peered":"2024-04-14T15:10:07.326206+0000","last_unstale":"2024-04-14T15:10:43.803967+0000","last_undegraded":"2024-04-14T15:10:43.803967+0000","last_fullsized":"2024-04-14T15:10:43.803967+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:37:00.130858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338804+0000","last_change":"2024-04-14T15:10:05.312258+0000","last_active":"2024-04-14T15:10:42.338804+0000","last_peered":"2024-04-14T15:10:42.338804+0000","last_clean":"2024-04-14T15:10:42.338804+0000","last_became_active":"2024-04-14T15:10:05.311943+0000","last_became_peered":"2024-04-14T15:10:05.311943+0000","last_unstale":"2024-04-14T15:10:42.338804+0000","last_undegraded":"2024-04-14T15:10:42.338804+0000","last_fullsized":"2024-04-14T15:10:42.338804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:14:45.667468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426810+0000","last_change":"2024-04-14T15:10:01.290547+0000","last_active":"2024-04-14T15:10:42.426810+0000","last_peered":"2024-04-14T15:10:42.426810+0000","last_clean":"2024-04-14T15:10:42.426810+0000","last_became_active":"2024-04-14T15:10:01.290140+0000","last_became_peered":"2024-04-14T15:10:01.290140+0000","last_unstale":"2024-04-14T15:10:42.426810+0000","last_undegraded":"2024-04-14T15:10:42.426810+0000","last_fullsized":"2024-04-14T15:10:42.426810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:44:13.843146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803366+0000","last_change":"2024-04-14T15:10:03.303911+0000","last_active":"2024-04-14T15:10:43.803366+0000","last_peered":"2024-04-14T15:10:43.803366+0000","last_clean":"2024-04-14T15:10:43.803366+0000","last_became_active":"2024-04-14T15:10:03.303344+0000","last_became_peered":"2024-04-14T15:10:03.303344+0000","last_unstale":"2024-04-14T15:10:43.803366+0000","last_undegraded":"2024-04-14T15:10:43.803366+0000","last_fullsized":"2024-04-14T15:10:43.803366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:16:56.934020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804561+0000","last_change":"2024-04-14T15:10:07.319951+0000","last_active":"2024-04-14T15:10:43.804561+0000","last_peered":"2024-04-14T15:10:43.804561+0000","last_clean":"2024-04-14T15:10:43.804561+0000","last_became_active":"2024-04-14T15:10:07.319553+0000","last_became_peered":"2024-04-14T15:10:07.319553+0000","last_unstale":"2024-04-14T15:10:43.804561+0000","last_undegraded":"2024-04-14T15:10:43.804561+0000","last_fullsized":"2024-04-14T15:10:43.804561+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:49:18.780378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":38,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426552+0000","last_change":"2024-04-14T15:10:05.310230+0000","last_active":"2024-04-14T15:10:42.426552+0000","last_peered":"2024-04-14T15:10:42.426552+0000","last_clean":"2024-04-14T15:10:42.426552+0000","last_became_active":"2024-04-14T15:10:01.287454+0000","last_became_peered":"2024-04-14T15:10:01.287454+0000","last_unstale":"2024-04-14T15:10:42.426552+0000","last_undegraded":"2024-04-14T15:10:42.426552+0000","last_fullsized":"2024-04-14T15:10:42.426552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:25:28.200138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020339440000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505521+0000","last_change":"2024-04-14T15:10:01.285979+0000","last_active":"2024-04-14T15:10:42.505521+0000","last_peered":"2024-04-14T15:10:42.505521+0000","last_clean":"2024-04-14T15:10:42.505521+0000","last_became_active":"2024-04-14T15:10:01.285625+0000","last_became_peered":"2024-04-14T15:10:01.285625+0000","last_unstale":"2024-04-14T15:10:42.505521+0000","last_undegraded":"2024-04-14T15:10:42.505521+0000","last_fullsized":"2024-04-14T15:10:42.505521+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:13:00.047342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":90,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344451+0000","last_change":"2024-04-14T15:10:03.302028+0000","last_active":"2024-04-14T15:11:16.344451+0000","last_peered":"2024-04-14T15:11:16.344451+0000","last_clean":"2024-04-14T15:11:16.344451+0000","last_became_active":"2024-04-14T15:10:03.301541+0000","last_became_peered":"2024-04-14T15:10:03.301541+0000","last_unstale":"2024-04-14T15:11:16.344451+0000","last_undegraded":"2024-04-14T15:11:16.344451+0000","last_fullsized":"2024-04-14T15:11:16.344451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:47:21.989515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804260+0000","last_change":"2024-04-14T15:10:05.314336+0000","last_active":"2024-04-14T15:10:43.804260+0000","last_peered":"2024-04-14T15:10:43.804260+0000","last_clean":"2024-04-14T15:10:43.804260+0000","last_became_active":"2024-04-14T15:10:05.314003+0000","last_became_peered":"2024-04-14T15:10:05.314003+0000","last_unstale":"2024-04-14T15:10:43.804260+0000","last_undegraded":"2024-04-14T15:10:43.804260+0000","last_fullsized":"2024-04-14T15:10:43.804260+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:31:52.222651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339439+0000","last_change":"2024-04-14T15:10:07.637085+0000","last_active":"2024-04-14T15:10:42.339439+0000","last_peered":"2024-04-14T15:10:42.339439+0000","last_clean":"2024-04-14T15:10:42.339439+0000","last_became_active":"2024-04-14T15:10:07.636768+0000","last_became_peered":"2024-04-14T15:10:07.636768+0000","last_unstale":"2024-04-14T15:10:42.339439+0000","last_undegraded":"2024-04-14T15:10:42.339439+0000","last_fullsized":"2024-04-14T15:10:42.339439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:10:35.431103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335782+0000","last_change":"2024-04-14T15:10:01.285910+0000","last_active":"2024-04-14T15:10:42.335782+0000","last_peered":"2024-04-14T15:10:42.335782+0000","last_clean":"2024-04-14T15:10:42.335782+0000","last_became_active":"2024-04-14T15:10:01.285450+0000","last_became_peered":"2024-04-14T15:10:01.285450+0000","last_unstale":"2024-04-14T15:10:42.335782+0000","last_undegraded":"2024-04-14T15:10:42.335782+0000","last_fullsized":"2024-04-14T15:10:42.335782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:31:51.247210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.338658+0000","last_change":"2024-04-14T15:10:05.302122+0000","last_active":"2024-04-14T15:10:42.338658+0000","last_peered":"2024-04-14T15:10:42.338658+0000","last_clean":"2024-04-14T15:10:42.338658+0000","last_became_active":"2024-04-14T15:10:01.284051+0000","last_became_peered":"2024-04-14T15:10:01.284051+0000","last_unstale":"2024-04-14T15:10:42.338658+0000","last_undegraded":"2024-04-14T15:10:42.338658+0000","last_fullsized":"2024-04-14T15:10:42.338658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:24.016649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086327099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":52,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336938+0000","last_change":"2024-04-14T15:10:03.299762+0000","last_active":"2024-04-14T15:10:42.336938+0000","last_peered":"2024-04-14T15:10:42.336938+0000","last_clean":"2024-04-14T15:10:42.336938+0000","last_became_active":"2024-04-14T15:10:03.299269+0000","last_became_peered":"2024-04-14T15:10:03.299269+0000","last_unstale":"2024-04-14T15:10:42.336938+0000","last_undegraded":"2024-04-14T15:10:42.336938+0000","last_fullsized":"2024-04-14T15:10:42.336938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:36:41.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339511+0000","last_change":"2024-04-14T15:10:05.316383+0000","last_active":"2024-04-14T15:10:42.339511+0000","last_peered":"2024-04-14T15:10:42.339511+0000","last_clean":"2024-04-14T15:10:42.339511+0000","last_became_active":"2024-04-14T15:10:05.316049+0000","last_became_peered":"2024-04-14T15:10:05.316049+0000","last_unstale":"2024-04-14T15:10:42.339511+0000","last_undegraded":"2024-04-14T15:10:42.339511+0000","last_fullsized":"2024-04-14T15:10:42.339511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:45:29.783005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427054+0000","last_change":"2024-04-14T15:10:07.322544+0000","last_active":"2024-04-14T15:10:42.427054+0000","last_peered":"2024-04-14T15:10:42.427054+0000","last_clean":"2024-04-14T15:10:42.427054+0000","last_became_active":"2024-04-14T15:10:07.322208+0000","last_became_peered":"2024-04-14T15:10:07.322208+0000","last_unstale":"2024-04-14T15:10:42.427054+0000","last_undegraded":"2024-04-14T15:10:42.427054+0000","last_fullsized":"2024-04-14T15:10:42.427054+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:49:13.037785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"70'128","reported_seq":268,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:44.420371+0000","last_change":"2024-04-14T15:09:52.891958+0000","last_active":"2024-04-14T15:10:44.420371+0000","last_peered":"2024-04-14T15:10:44.420371+0000","last_clean":"2024-04-14T15:10:44.420371+0000","last_became_active":"2024-04-14T15:09:52.891540+0000","last_became_peered":"2024-04-14T15:09:52.891540+0000","last_unstale":"2024-04-14T15:10:44.420371+0000","last_undegraded":"2024-04-14T15:10:44.420371+0000","last_fullsized":"2024-04-14T15:10:44.420371+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:07:49.498054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:07:49.498054+0000","last_clean_scrub_stamp":"2024-04-14T15:07:49.498054+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:20:20.335562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804400+0000","last_change":"2024-04-14T15:10:01.289421+0000","last_active":"2024-04-14T15:10:43.804400+0000","last_peered":"2024-04-14T15:10:43.804400+0000","last_clean":"2024-04-14T15:10:43.804400+0000","last_became_active":"2024-04-14T15:10:01.287764+0000","last_became_peered":"2024-04-14T15:10:01.287764+0000","last_unstale":"2024-04-14T15:10:43.804400+0000","last_undegraded":"2024-04-14T15:10:43.804400+0000","last_fullsized":"2024-04-14T15:10:43.804400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:29:21.212035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":64,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344605+0000","last_change":"2024-04-14T15:10:03.965170+0000","last_active":"2024-04-14T15:11:16.344605+0000","last_peered":"2024-04-14T15:11:16.344605+0000","last_clean":"2024-04-14T15:11:16.344605+0000","last_became_active":"2024-04-14T15:10:03.964799+0000","last_became_peered":"2024-04-14T15:10:03.964799+0000","last_unstale":"2024-04-14T15:11:16.344605+0000","last_undegraded":"2024-04-14T15:11:16.344605+0000","last_fullsized":"2024-04-14T15:11:16.344605+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:02:54.515909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339083+0000","last_change":"2024-04-14T15:10:05.318055+0000","last_active":"2024-04-14T15:10:42.339083+0000","last_peered":"2024-04-14T15:10:42.339083+0000","last_clean":"2024-04-14T15:10:42.339083+0000","last_became_active":"2024-04-14T15:10:05.317719+0000","last_became_peered":"2024-04-14T15:10:05.317719+0000","last_unstale":"2024-04-14T15:10:42.339083+0000","last_undegraded":"2024-04-14T15:10:42.339083+0000","last_fullsized":"2024-04-14T15:10:42.339083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:21:16.451753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335517+0000","last_change":"2024-04-14T15:10:07.331183+0000","last_active":"2024-04-14T15:10:42.335517+0000","last_peered":"2024-04-14T15:10:42.335517+0000","last_clean":"2024-04-14T15:10:42.335517+0000","last_became_active":"2024-04-14T15:10:07.330854+0000","last_became_peered":"2024-04-14T15:10:07.330854+0000","last_unstale":"2024-04-14T15:10:42.335517+0000","last_undegraded":"2024-04-14T15:10:42.335517+0000","last_fullsized":"2024-04-14T15:10:42.335517+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:37:30.621357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804562+0000","last_change":"2024-04-14T15:10:01.291751+0000","last_active":"2024-04-14T15:10:43.804562+0000","last_peered":"2024-04-14T15:10:43.804562+0000","last_clean":"2024-04-14T15:10:43.804562+0000","last_became_active":"2024-04-14T15:10:01.291431+0000","last_became_peered":"2024-04-14T15:10:01.291431+0000","last_unstale":"2024-04-14T15:10:43.804562+0000","last_undegraded":"2024-04-14T15:10:43.804562+0000","last_fullsized":"2024-04-14T15:10:43.804562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:38:45.110783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"68'5","reported_seq":115,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:21.669431+0000","last_change":"2024-04-14T15:10:05.309258+0000","last_active":"2024-04-14T15:11:21.669431+0000","last_peered":"2024-04-14T15:11:21.669431+0000","last_clean":"2024-04-14T15:11:21.669431+0000","last_became_active":"2024-04-14T15:10:01.290632+0000","last_became_peered":"2024-04-14T15:10:01.290632+0000","last_unstale":"2024-04-14T15:11:21.669431+0000","last_undegraded":"2024-04-14T15:11:21.669431+0000","last_fullsized":"2024-04-14T15:11:21.669431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:31:02.840870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043836199999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805335+0000","last_change":"2024-04-14T15:10:03.300802+0000","last_active":"2024-04-14T15:10:43.805335+0000","last_peered":"2024-04-14T15:10:43.805335+0000","last_clean":"2024-04-14T15:10:43.805335+0000","last_became_active":"2024-04-14T15:10:03.300483+0000","last_became_peered":"2024-04-14T15:10:03.300483+0000","last_unstale":"2024-04-14T15:10:43.805335+0000","last_undegraded":"2024-04-14T15:10:43.805335+0000","last_fullsized":"2024-04-14T15:10:43.805335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:18:51.955792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427473+0000","last_change":"2024-04-14T15:10:05.316548+0000","last_active":"2024-04-14T15:10:42.427473+0000","last_peered":"2024-04-14T15:10:42.427473+0000","last_clean":"2024-04-14T15:10:42.427473+0000","last_became_active":"2024-04-14T15:10:05.316218+0000","last_became_peered":"2024-04-14T15:10:05.316218+0000","last_unstale":"2024-04-14T15:10:42.427473+0000","last_undegraded":"2024-04-14T15:10:42.427473+0000","last_fullsized":"2024-04-14T15:10:42.427473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:17:18.940333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335523+0000","last_change":"2024-04-14T15:10:07.637225+0000","last_active":"2024-04-14T15:10:42.335523+0000","last_peered":"2024-04-14T15:10:42.335523+0000","last_clean":"2024-04-14T15:10:42.335523+0000","last_became_active":"2024-04-14T15:10:07.636840+0000","last_became_peered":"2024-04-14T15:10:07.636840+0000","last_unstale":"2024-04-14T15:10:42.335523+0000","last_undegraded":"2024-04-14T15:10:42.335523+0000","last_fullsized":"2024-04-14T15:10:42.335523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T19:05:08.889538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336182+0000","last_change":"2024-04-14T15:10:01.283024+0000","last_active":"2024-04-14T15:10:42.336182+0000","last_peered":"2024-04-14T15:10:42.336182+0000","last_clean":"2024-04-14T15:10:42.336182+0000","last_became_active":"2024-04-14T15:10:01.282699+0000","last_became_peered":"2024-04-14T15:10:01.282699+0000","last_unstale":"2024-04-14T15:10:42.336182+0000","last_undegraded":"2024-04-14T15:10:42.336182+0000","last_fullsized":"2024-04-14T15:10:42.336182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:40:30.016178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":61,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803930+0000","last_change":"2024-04-14T15:10:03.965436+0000","last_active":"2024-04-14T15:10:43.803930+0000","last_peered":"2024-04-14T15:10:43.803930+0000","last_clean":"2024-04-14T15:10:43.803930+0000","last_became_active":"2024-04-14T15:10:03.965035+0000","last_became_peered":"2024-04-14T15:10:03.965035+0000","last_unstale":"2024-04-14T15:10:43.803930+0000","last_undegraded":"2024-04-14T15:10:43.803930+0000","last_fullsized":"2024-04-14T15:10:43.803930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:03:59.112891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803197+0000","last_change":"2024-04-14T15:10:05.311617+0000","last_active":"2024-04-14T15:10:43.803197+0000","last_peered":"2024-04-14T15:10:43.803197+0000","last_clean":"2024-04-14T15:10:43.803197+0000","last_became_active":"2024-04-14T15:10:05.311258+0000","last_became_peered":"2024-04-14T15:10:05.311258+0000","last_unstale":"2024-04-14T15:10:43.803197+0000","last_undegraded":"2024-04-14T15:10:43.803197+0000","last_fullsized":"2024-04-14T15:10:43.803197+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T17:11:26.931324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.803906+0000","last_change":"2024-04-14T15:10:07.326925+0000","last_active":"2024-04-14T15:10:43.803906+0000","last_peered":"2024-04-14T15:10:43.803906+0000","last_clean":"2024-04-14T15:10:43.803906+0000","last_became_active":"2024-04-14T15:10:07.326573+0000","last_became_peered":"2024-04-14T15:10:07.326573+0000","last_unstale":"2024-04-14T15:10:43.803906+0000","last_undegraded":"2024-04-14T15:10:43.803906+0000","last_fullsized":"2024-04-14T15:10:43.803906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:04:29.291846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427022+0000","last_change":"2024-04-14T15:10:01.291008+0000","last_active":"2024-04-14T15:10:42.427022+0000","last_peered":"2024-04-14T15:10:42.427022+0000","last_clean":"2024-04-14T15:10:42.427022+0000","last_became_active":"2024-04-14T15:10:01.289754+0000","last_became_peered":"2024-04-14T15:10:01.289754+0000","last_unstale":"2024-04-14T15:10:42.427022+0000","last_undegraded":"2024-04-14T15:10:42.427022+0000","last_fullsized":"2024-04-14T15:10:42.427022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:07:50.687334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.335967+0000","last_change":"2024-04-14T15:10:03.301866+0000","last_active":"2024-04-14T15:10:42.335967+0000","last_peered":"2024-04-14T15:10:42.335967+0000","last_clean":"2024-04-14T15:10:42.335967+0000","last_became_active":"2024-04-14T15:10:03.299690+0000","last_became_peered":"2024-04-14T15:10:03.299690+0000","last_unstale":"2024-04-14T15:10:42.335967+0000","last_undegraded":"2024-04-14T15:10:42.335967+0000","last_fullsized":"2024-04-14T15:10:42.335967+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:28:13.401054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344458+0000","last_change":"2024-04-14T15:10:05.312319+0000","last_active":"2024-04-14T15:11:16.344458+0000","last_peered":"2024-04-14T15:11:16.344458+0000","last_clean":"2024-04-14T15:11:16.344458+0000","last_became_active":"2024-04-14T15:10:05.311910+0000","last_became_peered":"2024-04-14T15:10:05.311910+0000","last_unstale":"2024-04-14T15:11:16.344458+0000","last_undegraded":"2024-04-14T15:11:16.344458+0000","last_fullsized":"2024-04-14T15:11:16.344458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:46:38.273376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339013+0000","last_change":"2024-04-14T15:10:07.639889+0000","last_active":"2024-04-14T15:10:42.339013+0000","last_peered":"2024-04-14T15:10:42.339013+0000","last_clean":"2024-04-14T15:10:42.339013+0000","last_became_active":"2024-04-14T15:10:07.639568+0000","last_became_peered":"2024-04-14T15:10:07.639568+0000","last_unstale":"2024-04-14T15:10:42.339013+0000","last_undegraded":"2024-04-14T15:10:42.339013+0000","last_fullsized":"2024-04-14T15:10:42.339013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T18:44:17.822595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805627+0000","last_change":"2024-04-14T15:10:01.285916+0000","last_active":"2024-04-14T15:10:43.805627+0000","last_peered":"2024-04-14T15:10:43.805627+0000","last_clean":"2024-04-14T15:10:43.805627+0000","last_became_active":"2024-04-14T15:10:01.285596+0000","last_became_peered":"2024-04-14T15:10:01.285596+0000","last_unstale":"2024-04-14T15:10:43.805627+0000","last_undegraded":"2024-04-14T15:10:43.805627+0000","last_fullsized":"2024-04-14T15:10:43.805627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:18:17.934968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505453+0000","last_change":"2024-04-14T15:10:03.299486+0000","last_active":"2024-04-14T15:10:42.505453+0000","last_peered":"2024-04-14T15:10:42.505453+0000","last_clean":"2024-04-14T15:10:42.505453+0000","last_became_active":"2024-04-14T15:10:03.299136+0000","last_became_peered":"2024-04-14T15:10:03.299136+0000","last_unstale":"2024-04-14T15:10:42.505453+0000","last_undegraded":"2024-04-14T15:10:42.505453+0000","last_fullsized":"2024-04-14T15:10:42.505453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:42:31.753611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344449+0000","last_change":"2024-04-14T15:10:05.317136+0000","last_active":"2024-04-14T15:11:16.344449+0000","last_peered":"2024-04-14T15:11:16.344449+0000","last_clean":"2024-04-14T15:11:16.344449+0000","last_became_active":"2024-04-14T15:10:05.316824+0000","last_became_peered":"2024-04-14T15:10:05.316824+0000","last_unstale":"2024-04-14T15:11:16.344449+0000","last_undegraded":"2024-04-14T15:11:16.344449+0000","last_fullsized":"2024-04-14T15:11:16.344449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:48:05.709161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505410+0000","last_change":"2024-04-14T15:10:07.316986+0000","last_active":"2024-04-14T15:10:42.505410+0000","last_peered":"2024-04-14T15:10:42.505410+0000","last_clean":"2024-04-14T15:10:42.505410+0000","last_became_active":"2024-04-14T15:10:07.316647+0000","last_became_peered":"2024-04-14T15:10:07.316647+0000","last_unstale":"2024-04-14T15:10:42.505410+0000","last_undegraded":"2024-04-14T15:10:42.505410+0000","last_fullsized":"2024-04-14T15:10:42.505410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:35:59.372100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.805459+0000","last_change":"2024-04-14T15:10:01.289161+0000","last_active":"2024-04-14T15:10:43.805459+0000","last_peered":"2024-04-14T15:10:43.805459+0000","last_clean":"2024-04-14T15:10:43.805459+0000","last_became_active":"2024-04-14T15:10:01.288811+0000","last_became_peered":"2024-04-14T15:10:01.288811+0000","last_unstale":"2024-04-14T15:10:43.805459+0000","last_undegraded":"2024-04-14T15:10:43.805459+0000","last_fullsized":"2024-04-14T15:10:43.805459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:05:59.930810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":57,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804114+0000","last_change":"2024-04-14T15:10:03.965610+0000","last_active":"2024-04-14T15:10:43.804114+0000","last_peered":"2024-04-14T15:10:43.804114+0000","last_clean":"2024-04-14T15:10:43.804114+0000","last_became_active":"2024-04-14T15:10:03.965206+0000","last_became_peered":"2024-04-14T15:10:03.965206+0000","last_unstale":"2024-04-14T15:10:43.804114+0000","last_undegraded":"2024-04-14T15:10:43.804114+0000","last_fullsized":"2024-04-14T15:10:43.804114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:06:23.824599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505320+0000","last_change":"2024-04-14T15:10:05.318945+0000","last_active":"2024-04-14T15:10:42.505320+0000","last_peered":"2024-04-14T15:10:42.505320+0000","last_clean":"2024-04-14T15:10:42.505320+0000","last_became_active":"2024-04-14T15:10:05.317388+0000","last_became_peered":"2024-04-14T15:10:05.317388+0000","last_unstale":"2024-04-14T15:10:42.505320+0000","last_undegraded":"2024-04-14T15:10:42.505320+0000","last_fullsized":"2024-04-14T15:10:42.505320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:46:16.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.426806+0000","last_change":"2024-04-14T15:10:07.327189+0000","last_active":"2024-04-14T15:10:42.426806+0000","last_peered":"2024-04-14T15:10:42.426806+0000","last_clean":"2024-04-14T15:10:42.426806+0000","last_became_active":"2024-04-14T15:10:07.326830+0000","last_became_peered":"2024-04-14T15:10:07.326830+0000","last_unstale":"2024-04-14T15:10:42.426806+0000","last_undegraded":"2024-04-14T15:10:42.426806+0000","last_fullsized":"2024-04-14T15:10:42.426806+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T00:44:02.337817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344606+0000","last_change":"2024-04-14T15:10:05.312335+0000","last_active":"2024-04-14T15:11:16.344606+0000","last_peered":"2024-04-14T15:11:16.344606+0000","last_clean":"2024-04-14T15:11:16.344606+0000","last_became_active":"2024-04-14T15:10:05.312031+0000","last_became_peered":"2024-04-14T15:10:05.312031+0000","last_unstale":"2024-04-14T15:11:16.344606+0000","last_undegraded":"2024-04-14T15:11:16.344606+0000","last_fullsized":"2024-04-14T15:11:16.344606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:17:36.552106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804013+0000","last_change":"2024-04-14T15:10:01.284063+0000","last_active":"2024-04-14T15:10:43.804013+0000","last_peered":"2024-04-14T15:10:43.804013+0000","last_clean":"2024-04-14T15:10:43.804013+0000","last_became_active":"2024-04-14T15:10:01.283759+0000","last_became_peered":"2024-04-14T15:10:01.283759+0000","last_unstale":"2024-04-14T15:10:43.804013+0000","last_undegraded":"2024-04-14T15:10:43.804013+0000","last_fullsized":"2024-04-14T15:10:43.804013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:51:00.638086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.337127+0000","last_change":"2024-04-14T15:10:03.300367+0000","last_active":"2024-04-14T15:10:42.337127+0000","last_peered":"2024-04-14T15:10:42.337127+0000","last_clean":"2024-04-14T15:10:42.337127+0000","last_became_active":"2024-04-14T15:10:03.299175+0000","last_became_peered":"2024-04-14T15:10:03.299175+0000","last_unstale":"2024-04-14T15:10:42.337127+0000","last_undegraded":"2024-04-14T15:10:42.337127+0000","last_fullsized":"2024-04-14T15:10:42.337127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:53:23.786101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"68'1","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804669+0000","last_change":"2024-04-14T15:10:07.637118+0000","last_active":"2024-04-14T15:10:43.804669+0000","last_peered":"2024-04-14T15:10:43.804669+0000","last_clean":"2024-04-14T15:10:43.804669+0000","last_became_active":"2024-04-14T15:10:07.636721+0000","last_became_peered":"2024-04-14T15:10:07.636721+0000","last_unstale":"2024-04-14T15:10:43.804669+0000","last_undegraded":"2024-04-14T15:10:43.804669+0000","last_fullsized":"2024-04-14T15:10:43.804669+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:30:07.542573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.427383+0000","last_change":"2024-04-14T15:10:05.316726+0000","last_active":"2024-04-14T15:10:42.427383+0000","last_peered":"2024-04-14T15:10:42.427383+0000","last_clean":"2024-04-14T15:10:42.427383+0000","last_became_active":"2024-04-14T15:10:05.315634+0000","last_became_peered":"2024-04-14T15:10:05.315634+0000","last_unstale":"2024-04-14T15:10:42.427383+0000","last_undegraded":"2024-04-14T15:10:42.427383+0000","last_fullsized":"2024-04-14T15:10:42.427383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T02:29:39.819859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505347+0000","last_change":"2024-04-14T15:10:01.289850+0000","last_active":"2024-04-14T15:10:42.505347+0000","last_peered":"2024-04-14T15:10:42.505347+0000","last_clean":"2024-04-14T15:10:42.505347+0000","last_became_active":"2024-04-14T15:10:01.289516+0000","last_became_peered":"2024-04-14T15:10:01.289516+0000","last_unstale":"2024-04-14T15:10:42.505347+0000","last_undegraded":"2024-04-14T15:10:42.505347+0000","last_fullsized":"2024-04-14T15:10:42.505347+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T20:31:39.917053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":44,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505979+0000","last_change":"2024-04-14T15:10:03.306367+0000","last_active":"2024-04-14T15:10:42.505979+0000","last_peered":"2024-04-14T15:10:42.505979+0000","last_clean":"2024-04-14T15:10:42.505979+0000","last_became_active":"2024-04-14T15:10:03.306075+0000","last_became_peered":"2024-04-14T15:10:03.306075+0000","last_unstale":"2024-04-14T15:10:42.505979+0000","last_undegraded":"2024-04-14T15:10:42.505979+0000","last_fullsized":"2024-04-14T15:10:42.505979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T16:59:31.179931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.339719+0000","last_change":"2024-04-14T15:10:07.316092+0000","last_active":"2024-04-14T15:10:42.339719+0000","last_peered":"2024-04-14T15:10:42.339719+0000","last_clean":"2024-04-14T15:10:42.339719+0000","last_became_active":"2024-04-14T15:10:07.315761+0000","last_became_peered":"2024-04-14T15:10:07.315761+0000","last_unstale":"2024-04-14T15:10:42.339719+0000","last_undegraded":"2024-04-14T15:10:42.339719+0000","last_fullsized":"2024-04-14T15:10:42.339719+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:06:31.680274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.336342+0000","last_change":"2024-04-14T15:10:05.310948+0000","last_active":"2024-04-14T15:10:42.336342+0000","last_peered":"2024-04-14T15:10:42.336342+0000","last_clean":"2024-04-14T15:10:42.336342+0000","last_became_active":"2024-04-14T15:10:05.310544+0000","last_became_peered":"2024-04-14T15:10:05.310544+0000","last_unstale":"2024-04-14T15:10:42.336342+0000","last_undegraded":"2024-04-14T15:10:42.336342+0000","last_fullsized":"2024-04-14T15:10:42.336342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T22:13:46.271225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.802982+0000","last_change":"2024-04-14T15:10:01.284704+0000","last_active":"2024-04-14T15:10:43.802982+0000","last_peered":"2024-04-14T15:10:43.802982+0000","last_clean":"2024-04-14T15:10:43.802982+0000","last_became_active":"2024-04-14T15:10:01.284069+0000","last_became_peered":"2024-04-14T15:10:01.284069+0000","last_unstale":"2024-04-14T15:10:43.802982+0000","last_undegraded":"2024-04-14T15:10:43.802982+0000","last_fullsized":"2024-04-14T15:10:43.802982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:27:02.834976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":53,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506065+0000","last_change":"2024-04-14T15:10:03.305992+0000","last_active":"2024-04-14T15:10:42.506065+0000","last_peered":"2024-04-14T15:10:42.506065+0000","last_clean":"2024-04-14T15:10:42.506065+0000","last_became_active":"2024-04-14T15:10:03.305666+0000","last_became_peered":"2024-04-14T15:10:03.305666+0000","last_unstale":"2024-04-14T15:10:42.506065+0000","last_undegraded":"2024-04-14T15:10:42.506065+0000","last_fullsized":"2024-04-14T15:10:42.506065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:22:48.366305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.506119+0000","last_change":"2024-04-14T15:10:07.329171+0000","last_active":"2024-04-14T15:10:42.506119+0000","last_peered":"2024-04-14T15:10:42.506119+0000","last_clean":"2024-04-14T15:10:42.506119+0000","last_became_active":"2024-04-14T15:10:07.328890+0000","last_became_peered":"2024-04-14T15:10:07.328890+0000","last_unstale":"2024-04-14T15:10:42.506119+0000","last_undegraded":"2024-04-14T15:10:42.506119+0000","last_fullsized":"2024-04-14T15:10:42.506119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:06.297909+0000","last_clean_scrub_stamp":"2024-04-14T15:10:06.297909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T23:56:32.325594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":46,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:11:16.344518+0000","last_change":"2024-04-14T15:10:05.313473+0000","last_active":"2024-04-14T15:11:16.344518+0000","last_peered":"2024-04-14T15:11:16.344518+0000","last_clean":"2024-04-14T15:11:16.344518+0000","last_became_active":"2024-04-14T15:10:05.313006+0000","last_became_peered":"2024-04-14T15:10:05.313006+0000","last_unstale":"2024-04-14T15:11:16.344518+0000","last_undegraded":"2024-04-14T15:11:16.344518+0000","last_fullsized":"2024-04-14T15:11:16.344518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:04.287136+0000","last_clean_scrub_stamp":"2024-04-14T15:10:04.287136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T21:09:23.666868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:43.804682+0000","last_change":"2024-04-14T15:10:01.288658+0000","last_active":"2024-04-14T15:10:43.804682+0000","last_peered":"2024-04-14T15:10:43.804682+0000","last_clean":"2024-04-14T15:10:43.804682+0000","last_became_active":"2024-04-14T15:10:01.288352+0000","last_became_peered":"2024-04-14T15:10:01.288352+0000","last_unstale":"2024-04-14T15:10:43.804682+0000","last_undegraded":"2024-04-14T15:10:43.804682+0000","last_fullsized":"2024-04-14T15:10:43.804682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:00.263992+0000","last_clean_scrub_stamp":"2024-04-14T15:10:00.263992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-16T01:11:10.332314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-14T15:10:42.505733+0000","last_change":"2024-04-14T15:10:03.302665+0000","last_active":"2024-04-14T15:10:42.505733+0000","last_peered":"2024-04-14T15:10:42.505733+0000","last_clean":"2024-04-14T15:10:42.505733+0000","last_became_active":"2024-04-14T15:10:03.302422+0000","last_became_peered":"2024-04-14T15:10:03.302422+0000","last_unstale":"2024-04-14T15:10:42.505733+0000","last_undegraded":"2024-04-14T15:10:42.505733+0000","last_fullsized":"2024-04-14T15:10:42.505733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-14T15:10:02.273374+0000","last_clean_scrub_stamp":"2024-04-14T15:10:02.273374+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-15T15:33:51.914041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712720,"statfs":{"total":95995035648,"available":95961825280,"internally_reserved":0,"allocated":1282048,"data_stored":857385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716892,"statfs":{"total":95995035648,"available":95966097408,"internally_reserved":0,"allocated":1269760,"data_stored":856782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31900,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713252,"statfs":{"total":95995035648,"available":95962370048,"internally_reserved":0,"allocated":737280,"data_stored":268661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28276,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716876,"statfs":{"total":95995035648,"available":95966081024,"internally_reserved":0,"allocated":1286144,"data_stored":857127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717480,"statfs":{"total":95995035648,"available":95966699520,"internally_reserved":0,"allocated":667648,"data_stored":266516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717432,"statfs":{"total":95995035648,"available":95966650368,"internally_reserved":0,"allocated":716800,"data_stored":268216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713288,"statfs":{"total":95995035648,"available":95962406912,"internally_reserved":0,"allocated":700416,"data_stored":267481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717472,"statfs":{"total":95995035648,"available":95966691328,"internally_reserved":0,"allocated":675840,"data_stored":266304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-14T15:11:28.568 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-14T15:11:28.568 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-14T15:11:28.568 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-14T15:11:28.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888 shell --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 -- ceph health --format=json 2024-04-14T15:11:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/mon.a/config 2024-04-14T15:11:29.109 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:29 smithi045 ceph-mon[37795]: from='client.34643 v1:172.21.15.45:0/3012565086' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:29.109 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:29 smithi045 ceph-mon[29624]: from='client.34643 v1:172.21.15.45:0/3012565086' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:29 smithi071 ceph-mon[33150]: from='client.34643 v1:172.21.15.45:0/3012565086' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-14T15:11:30.342 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:30 smithi045 ceph-mon[29624]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:30.343 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:30 smithi045 ceph-mon[37795]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:30.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:30 smithi071 ceph-mon[33150]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.444+0000 7fecac9c7700 1 -- 172.21.15.45:0/2264300012 >> v1:172.21.15.45:6789/0 conn(0x7feca810b730 legacy=0x7feca810db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.444+0000 7fecac9c7700 1 -- 172.21.15.45:0/2264300012 shutdown_connections 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.444+0000 7fecac9c7700 1 -- 172.21.15.45:0/2264300012 >> 172.21.15.45:0/2264300012 conn(0x7feca80fe3d0 msgr2=0x7feca81007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.444+0000 7fecac9c7700 1 -- 172.21.15.45:0/2264300012 shutdown_connections 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.444+0000 7fecac9c7700 1 -- 172.21.15.45:0/2264300012 wait complete. 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fecac9c7700 1 Processor -- start 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fecac9c7700 1 -- start start 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fecac9c7700 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7feca81a78c0 con 0x7feca8108880 2024-04-14T15:11:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fecac9c7700 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7feca81a8ae0 con 0x7feca810b730 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fecac9c7700 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7feca81a9d20 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7feca659c700 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feca8108880 0x7feca810ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51568/0 (socket says 172.21.15.45:51568) 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7feca659c700 1 -- 172.21.15.45:0/1818192027 learned_addr learned my addr 172.21.15.45:0/1818192027 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364643271 0 0) 0x7feca81a9d20 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec900035b0 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454977664 0 0) 0x7feca81a78c0 con 0x7feca8108880 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feca81a9d20 con 0x7feca8108880 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1253547914 0 0) 0x7feca81a8ae0 con 0x7feca810b730 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.445+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feca81a78c0 con 0x7feca810b730 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1240061139 0 0) 0x7fec900035b0 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feca81a8ae0 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7feca0001bf0 con 0x7feca810f300 2024-04-14T15:11:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2089461639 0 0) 0x7feca81a9d20 con 0x7feca8108880 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fec900035b0 con 0x7feca8108880 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fec980018b0 con 0x7feca8108880 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3191454725 0 0) 0x7feca81a78c0 con 0x7feca810b730 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7feca81a9d20 con 0x7feca810b730 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fec9c001370 con 0x7feca810b730 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 968234076 0 0) 0x7feca81a8ae0 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 >> v1:172.21.15.71:6789/0 conn(0x7feca810b730 legacy=0x7feca81a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 >> v1:172.21.15.45:6789/0 conn(0x7feca8108880 legacy=0x7feca810ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca81aaf60 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7feca81a7af0 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7feca81a7fb0 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7feca0003e90 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.446+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7feca0004e50 con 0x7feca810f300 2024-04-14T15:11:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.447+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec88005150 con 0x7feca810f300 2024-04-14T15:11:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.447+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7feca0003a00 con 0x7feca810f300 2024-04-14T15:11:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.448+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(70..70 src has 1..70) ==== 6172+0+0 (unknown 334351988 0 0) 0x7feca0089430 con 0x7feca810f300 2024-04-14T15:11:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.453+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7feca0056670 con 0x7feca810f300 2024-04-14T15:11:30.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.709+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fec88005440 con 0x7feca810f300 2024-04-14T15:11:30.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.709+0000 7fec977fe700 1 -- 172.21.15.45:0/1818192027 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7feca005a320 con 0x7feca810f300 2024-04-14T15:11:30.710 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-14T15:11:30.710 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-14T15:11:30.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.711+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 >> v1:172.21.15.45:6800/1718216366 conn(0x7fec9006faa0 legacy=0x7fec90071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.711+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 >> v1:172.21.15.45:6790/0 conn(0x7feca810f300 legacy=0x7feca81a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:11:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.711+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 shutdown_connections 2024-04-14T15:11:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.711+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 >> 172.21.15.45:0/1818192027 conn(0x7feca80fe3d0 msgr2=0x7feca8111f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:11:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.711+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 shutdown_connections 2024-04-14T15:11:30.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-14T15:11:30.712+0000 7fecac9c7700 1 -- 172.21.15.45:0/1818192027 wait complete. 2024-04-14T15:11:31.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1818192027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-14T15:11:31.389 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1818192027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-14T15:11:31.389 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1818192027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-14T15:11:31.417 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-14T15:11:31.417 INFO:tasks.cephadm:Setup complete, yielding 2024-04-14T15:11:31.417 INFO:teuthology.run_tasks:Running task workunit... 2024-04-14T15:11:31.431 INFO:tasks.workunit:Pulling workunits from ref 05c9d46e7de5db2cb434c177a6bbfc56f472d09b 2024-04-14T15:11:31.431 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-14T15:11:31.431 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-14T15:11:31.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T15:11:31.447 INFO:teuthology.orchestra.run.smithi045.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-14T15:11:31.448 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-14T15:11:31.502 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-14T15:11:31.503 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-14T15:11:31.557 INFO:tasks.workunit:timeout=3h 2024-04-14T15:11:31.557 INFO:tasks.workunit:cleanup=True 2024-04-14T15:11:31.558 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 05c9d46e7de5db2cb434c177a6bbfc56f472d09b 2024-04-14T15:11:31.613 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-14T15:11:32.329 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:32 smithi071 ceph-mon[33150]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:32 smithi045 ceph-mon[29624]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:32 smithi045 ceph-mon[37795]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:33.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:33.239Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00187191s 2024-04-14T15:11:33.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:34.037 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:33 smithi071 systemd[1]: Starting Ceph grafana.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-mon[33150]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:34.357 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 podman[63833]: 2024-04-14 15:11:33.94040036 +0000 UTC m=+0.018007938 image pull 2bacad6d85d802cd176c146528adcc0e47a8a83e94bc4a8b7ba1a9b9ef2c65e7 quay.io/ceph/grafana:9.4.12 2024-04-14T15:11:34.357 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 podman[63833]: 2024-04-14 15:11:34.073863179 +0000 UTC m=+0.151470752 container create 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:11:34.357 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 podman[63833]: 2024-04-14 15:11:34.276256663 +0000 UTC m=+0.353864241 container init 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:11:34.357 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 podman[63833]: 2024-04-14 15:11:34.27869337 +0000 UTC m=+0.356300954 container start 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:11:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:34 smithi045 ceph-mon[37795]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:34 smithi045 ceph-mon[29624]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:34.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 bash[63833]: 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 2024-04-14T15:11:34.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 systemd[1]: Started Ceph grafana.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:34.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431573546Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-04-14T15:11:34.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431752333Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-04-14T15:11:34.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431765973Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431769711Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.43177246Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431775678Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431778299Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431780752Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431783261Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431786303Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431789478Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431791971Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431794411Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431796958Z level=info msg="Path Home" path=/usr/share/grafana 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431799569Z level=info msg="Path Data" path=/var/lib/grafana 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431802089Z level=info msg="Path Logs" path=/var/log/grafana 2024-04-14T15:11:34.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.43180437Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-04-14T15:11:34.610 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431806888Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-04-14T15:11:34.610 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=settings t=2024-04-14T15:11:34.431809451Z level=info msg="App mode production" 2024-04-14T15:11:34.610 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore t=2024-04-14T15:11:34.431854687Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-04-14T15:11:34.610 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore t=2024-04-14T15:11:34.431867847Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-04-14T15:11:34.610 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.432697624Z level=info msg="Starting DB migrations" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.432768627Z level=info msg="Executing migration" id="create migration_log table" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.433283625Z level=info msg="Executing migration" id="create user table" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.433690371Z level=info msg="Executing migration" id="add unique index user.login" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.43671043Z level=info msg="Executing migration" id="add unique index user.email" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.439416431Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.442094076Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.444878847Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.447598715Z level=info msg="Executing migration" id="create user table v2" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.44799774Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.450861395Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.451438761Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-04-14T15:11:34.611 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.453470215Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.453869597Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.456206567Z level=info msg="Executing migration" id="Update user table charset" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.456508743Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.458998402Z level=info msg="Executing migration" id="Add missing user data" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.46173535Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.464493752Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.467190747Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.469811832Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.472488584Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.475371546Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.475895497Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.478673967Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.479580238Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.480334824Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-04-14T15:11:34.612 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.481186506Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.48202121Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.482868486Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.483716818Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.484146158Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.484980642Z level=info msg="Executing migration" id="create temp_user v2" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.485366159Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.485738029Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.486157706Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.486527918Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-04-14T15:11:34.613 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.486991813Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.487397344Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.488238243Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.488591182Z level=info msg="Executing migration" id="create star table" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.489035052Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.489934811Z level=info msg="Executing migration" id="create org table v1" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.490903785Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.491739697Z level=info msg="Executing migration" id="create org_user table v1" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.492495774Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-04-14T15:11:34.614 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.493358806Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.494232537Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.495141106Z level=info msg="Executing migration" id="Update org table charset" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.495933397Z level=info msg="Executing migration" id="Update org_user table charset" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.496816754Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.497670573Z level=info msg="Executing migration" id="create dashboard table" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.498626754Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.49938447Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-04-14T15:11:34.615 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.50028806Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.50115225Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.501965346Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.502937819Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.504104856Z level=info msg="Executing migration" id="create dashboard v2" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.504900796Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.505327424Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.507976474Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.510904837Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.513617798Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-04-14T15:11:34.616 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.516585855Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.519612309Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.522348944Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.525138604Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.528124045Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.530974236Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.533566905Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.536559605Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.539418194Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.542001144Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.544809158Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.547629082Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.550354979Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.553072019Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.555992508Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-04-14T15:11:34.617 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.558575347Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.561250863Z level=info msg="Executing migration" id="Update dashboard title length" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.564040672Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.566813952Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.569461291Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.57231016Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.575138493Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.577720473Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.580535451Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.583151993Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.585889631Z level=info msg="Executing migration" id="Add check_sum column" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.588589411Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.591235731Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.594010266Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-04-14T15:11:34.618 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.596712433Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.599325045Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.600295645Z level=info msg="Executing migration" id="create data_source table" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.600998069Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.601861798Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.602723081Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.603681785Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.604411364Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.605631364Z level=info msg="Executing migration" id="create data_source table v2" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.606610922Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.607327223Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.608292394Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.609049643Z level=info msg="Executing migration" id="Add column with_credentials" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.609900892Z level=info msg="Executing migration" id="Add secure json data column" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.610717197Z level=info msg="Executing migration" id="Update data_source table charset" 2024-04-14T15:11:34.619 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.611577368Z level=info msg="Executing migration" id="Update initial version to 1" 2024-04-14T15:11:34.620 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.612390229Z level=info msg="Executing migration" id="Add read_only data column" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.613159775Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.614044022Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.614937535Z level=info msg="Executing migration" id="Add uid column" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.615785578Z level=info msg="Executing migration" id="Update uid value" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.616628063Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-04-14T15:11:34.859 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.617509264Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.618382296Z level=info msg="Executing migration" id="create api_key table" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.619395837Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.620115906Z level=info msg="Executing migration" id="add index api_key.key" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.621093571Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.621859079Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.622831052Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.6235474Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.623940162Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.625202434Z level=info msg="Executing migration" id="create api_key table v2" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.626164769Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.626887063Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.627281766Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.627681137Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.628006075Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-04-14T15:11:34.860 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.628365559Z level=info msg="Executing migration" id="Update api_key table charset" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.628608142Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.629159629Z level=info msg="Executing migration" id="Add service account foreign key" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.629709429Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.630017961Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.630571791Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.631162659Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.631528489Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.631844707Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.632215996Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.632607287Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.633002278Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.633390004Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.633638751Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.633916893Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-04-14T15:11:34.861 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.634518561Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-04-14T15:11:34.862 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.635117247Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-04-14T15:11:34.862 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.635410698Z level=info msg="Executing migration" id="create quota table v1" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.635783705Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.636197613Z level=info msg="Executing migration" id="Update quota table charset" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.636443751Z level=info msg="Executing migration" id="create plugin_setting table" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.63681645Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.637214286Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.637805029Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.638094958Z level=info msg="Executing migration" id="create session table" 2024-04-14T15:11:34.863 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.638494279Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.63873975Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.639026286Z level=info msg="Executing migration" id="create playlist table v2" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.639428111Z level=info msg="Executing migration" id="create playlist item table v2" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.63978966Z level=info msg="Executing migration" id="Update playlist table charset" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.64006389Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.640305719Z level=info msg="Executing migration" id="drop preferences table v2" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.6405544Z level=info msg="Executing migration" id="drop preferences table v3" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.640826598Z level=info msg="Executing migration" id="create preferences table v3" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.641353707Z level=info msg="Executing migration" id="Update preferences table charset" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.64179188Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-04-14T15:11:34.864 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.642429623Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.642689904Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.643313688Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.643923926Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.644204201Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.644599247Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.64508123Z level=info msg="Executing migration" id="create alert table v1" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.645897402Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.646336186Z level=info msg="Executing migration" id="add index alert state" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.649289786Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.652198364Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.654775647Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.655395664Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.658217315Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.659950427Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-04-14T15:11:34.865 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.661014941Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.663526149Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.664066677Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.666858849Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.669562164Z level=info msg="Executing migration" id="Add column is_default" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.672280211Z level=info msg="Executing migration" id="Add column frequency" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.673052539Z level=info msg="Executing migration" id="Add column send_reminder" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.675827079Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.678532767Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.68153834Z level=info msg="Executing migration" id="Update alert table charset" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.681897607Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.684707826Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.687377068Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.690188976Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-04-14T15:11:34.866 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.690657833Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.693526588Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.696303368Z level=info msg="Executing migration" id="Add for to alert table" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.699007606Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.699827661Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.702391792Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.705236403Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.707927826Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.708726768Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.711399826Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.714264295Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.71691516Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.717265261Z level=info msg="Executing migration" id="create annotation table v5" 2024-04-14T15:11:34.867 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.720080917Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.720760661Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.721696853Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.722541379Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.723197599Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.723961617Z level=info msg="Executing migration" id="Update annotation table charset" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.724824414Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.72563222Z level=info msg="Executing migration" id="Drop category_id index" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.726592641Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.727452766Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.728305197Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.728783068Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.729585752Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.731586712Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-04-14T15:11:34.868 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.732360335Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.732829872Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.733574214Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.734524568Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.735246128Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.736075763Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.736922647Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.737930276Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.73862049Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.738942628Z level=info msg="Executing migration" id="Add epoch_end column" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.739829119Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.740830201Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.7414328Z level=info msg="Executing migration" id="Move region to single row" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.741786175Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.742248249Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-04-14T15:11:34.869 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.742660641Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.743149653Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.743593729Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.74443967Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.744917276Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.745905566Z level=info msg="Executing migration" id="create test_data table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.746552991Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.746958902Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.74740835Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.750393953Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.753191873Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.75578446Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.756124907Z level=info msg="Executing migration" id="create team table" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.759054104Z level=info msg="Executing migration" id="add index team.org_id" 2024-04-14T15:11:34.870 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.761887003Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.764572029Z level=info msg="Executing migration" id="create team member table" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.765020266Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.767714975Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.770406779Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.773175279Z level=info msg="Executing migration" id="Add column email to team table" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.774045175Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.776698801Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.779634594Z level=info msg="Executing migration" id="create dashboard acl table" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.782209276Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.782699769Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.785423293Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.786102161Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.788791156Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.789245351Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-04-14T15:11:34.871 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.789692924Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.790161632Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.790584534Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.791427077Z level=info msg="Executing migration" id="create tag table" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.794606462Z level=info msg="Executing migration" id="add index tag.key_value" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.797484927Z level=info msg="Executing migration" id="create login attempt table" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.800218324Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.80295053Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.805791089Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.808774737Z level=info msg="Executing migration" id="create login_attempt v2" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.81168997Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.814547246Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.81729553Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.820241506Z level=info msg="Executing migration" id="create user auth table" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.823221304Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-04-14T15:11:34.872 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.8261212Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.82893638Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.831554189Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.8343817Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.837189292Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.840133404Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.842875329Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.845563367Z level=info msg="Executing migration" id="create server_lock table" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.848314319Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.851083148Z level=info msg="Executing migration" id="create user auth token table" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.85379083Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.856596561Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.859350781Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-04-14T15:11:34.873 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.861980313Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-04-14T15:11:35.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:35.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.110 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.864757198Z level=info msg="Executing migration" id="create cache_data table" 2024-04-14T15:11:35.110 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.867433653Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-04-14T15:11:35.110 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.870201656Z level=info msg="Executing migration" id="create short_url table v1" 2024-04-14T15:11:35.110 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.872830044Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-04-14T15:11:35.110 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.875675023Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.878204204Z level=info msg="Executing migration" id="delete alert_definition table" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.88106723Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.883658117Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.88651658Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.889179966Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.890123687Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.890845537Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.891692585Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.892495834Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.893322696Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.894244983Z level=info msg="Executing migration" id="drop alert_definition table" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.895140225Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-04-14T15:11:35.111 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.895883082Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.896849266Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.897687375Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.898402703Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.89923414Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.899704945Z level=info msg="Executing migration" id="create alert_instance table" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.900128051Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.901044776Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.901555966Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.902435752Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.903309916Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-04-14T15:11:35.112 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.904166655Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.908587488Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.912993483Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.913849853Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.914794651Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.915813187Z level=info msg="Executing migration" id="create alert_rule table" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.916874382Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.917592718Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.91858268Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.919289307Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.920327834Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.921332062Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.922270437Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-04-14T15:11:35.113 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.923258857Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.924133617Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.925108904Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.926273987Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.929348102Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.93216895Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.934908175Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.937837673Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.94065415Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.943415956Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.946173255Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-04-14T15:11:35.114 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.949236797Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.952226376Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.954925301Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.957790307Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.960651712Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.963585722Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.96629682Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.969154481Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.971921239Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.974865045Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.977713124Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.980336459Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.983125553Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.985908986Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.988605364Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.991197317Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-04-14T15:11:35.115 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.99402867Z level=info msg="Executing migration" id="create provenance_type table" 2024-04-14T15:11:35.116 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.996788705Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-04-14T15:11:35.116 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:34.999644875Z level=info msg="Executing migration" id="create alert_image table" 2024-04-14T15:11:35.116 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.002325347Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.005057144Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.007687126Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.010396615Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.013061815Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.013132159Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.01580438Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.018588319Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.021411308Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.021566426Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-04-14T15:11:35.117 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.021674222Z level=info msg="alerts found to migrate" alerts=0 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.024128016Z level=info msg="Executing migration" id="create library_element table v1" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.024843037Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.025862862Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.026549176Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.027345461Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.028193075Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.029035621Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.029951001Z level=info msg="Executing migration" id="create data_keys table" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.03078403Z level=info msg="Executing migration" id="create secrets table" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.031631307Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.037164434Z level=info msg="Executing migration" id="add name column into data_keys" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.038308761Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.039128766Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-04-14T15:11:35.118 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.044764396Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.05040829Z level=info msg="Executing migration" id="create kv_store table v1" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.051238741Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.052013075Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.052905508Z level=info msg="Executing migration" id="create permission table" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.053780532Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.054776523Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.055477515Z level=info msg="Executing migration" id="create role table" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.056327329Z level=info msg="Executing migration" id="add column display_name" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.057441024Z level=info msg="Executing migration" id="add column group_name" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.058497142Z level=info msg="Executing migration" id="add index role.org_id" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.059376543Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.059831403Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.060319452Z level=info msg="Executing migration" id="create team role table" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.060698819Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.06113921Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-04-14T15:11:35.119 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.061597699Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.062048932Z level=info msg="Executing migration" id="create user role table" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.062465729Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.062920037Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.06340461Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.063860076Z level=info msg="Executing migration" id="create builtin role table" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.064305356Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.064781108Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.065265743Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.066425961Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.066950632Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-04-14T15:11:35.120 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.067422613Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.067824209Z level=info msg="Executing migration" id="add unique index role.uid" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.068303157Z level=info msg="Executing migration" id="create seed assignment table" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.068704453Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.069188915Z level=info msg="Executing migration" id="add column hidden to role table" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.070345925Z level=info msg="Executing migration" id="create query_history table v1" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.070765813Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.071245099Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.071511125Z level=info msg="Executing migration" id="teams permissions migration" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.071926606Z level=info msg="Executing migration" id="dashboard permissions" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.072420438Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.072783034Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.073096918Z level=info msg="Executing migration" id="alerting notification permissions" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.073431832Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-04-14T15:11:35.121 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.073813275Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.074263294Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.075433585Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.075731583Z level=info msg="Executing migration" id="create correlation table v1" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.076196722Z level=info msg="Executing migration" id="add index correlations.uid" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.077027395Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.077500962Z level=info msg="Executing migration" id="add correlation config column" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.078649534Z level=info msg="Executing migration" id="create entity_events table" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.07906009Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-04-14T15:11:35.122 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.079480727Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.079557802Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.080012838Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.080093891Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.080361838Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.081116534Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.081584854Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.08204668Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.082893414Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.08375773Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.084628983Z level=info msg="Executing migration" id="Drop public config table" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.085391943Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.086244173Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.086924202Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.087395847Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-04-14T15:11:35.123 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.08825672Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.092058209Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.093278186Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.094471587Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.095287806Z level=info msg="Executing migration" id="add share column" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.096629658Z level=info msg="Executing migration" id="create default alerting folders" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.097367577Z level=info msg="Executing migration" id="create file table" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.098355975Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.099027694Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.099841175Z level=info msg="Executing migration" id="create file_meta table" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.100264589Z level=info msg="Executing migration" id="file table idx: path key" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.101120109Z level=info msg="Executing migration" id="set path collation in file table" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.10194119Z level=info msg="Executing migration" id="managed permissions migration" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.102714366Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-04-14T15:11:35.124 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.103555088Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.10440771Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.105883296Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.106199329Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.106992686Z level=info msg="Executing migration" id="update group index for alert rules" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.107843476Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.108692345Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.109631679Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-04-14T15:11:35.125 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.111546223Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-04-14T15:11:35.434 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.435 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:35 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:35.607 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.114205807Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-04-14T15:11:35.607 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.11707899Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-04-14T15:11:35.607 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.133143223Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-04-14T15:11:35.607 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.135772774Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-04-14T15:11:35.607 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.138474197Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.142720244Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.14554649Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.148408762Z level=info msg="Executing migration" id="create folder table" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.150831831Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.153545135Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.156353128Z level=info msg="Executing migration" id="Update folder title length" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.159030623Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=migrator t=2024-04-14T15:11:35.161768251Z level=info msg="migrations completed" performed=485 skipped=0 duration=729.026848ms 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore t=2024-04-14T15:11:35.162134396Z level=info msg="Created default organization" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=plugin.finder t=2024-04-14T15:11:35.195430847Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=secrets t=2024-04-14T15:11:35.195532383Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=query_data t=2024-04-14T15:11:35.198923008Z level=info msg="Query Service initialization" 2024-04-14T15:11:35.608 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=live.push_http t=2024-04-14T15:11:35.201448353Z level=info msg="Live Push Gateway initialization" 2024-04-14T15:11:35.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:35 smithi045 bash[72858]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=infra.usagestats.collector t=2024-04-14T15:11:35.799525582Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=provisioning.datasources t=2024-04-14T15:11:35.800466099Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=provisioning.datasources t=2024-04-14T15:11:35.811139647Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=provisioning.alerting t=2024-04-14T15:11:35.821484107Z level=info msg="starting to provision alerting" 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=provisioning.alerting t=2024-04-14T15:11:35.821493724Z level=info msg="finished to provision alerting" 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=ngalert.state.manager t=2024-04-14T15:11:35.821562287Z level=info msg="Warming state cache for startup" 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=grafanaStorageLogger t=2024-04-14T15:11:35.821710295Z level=info msg="storage starting" 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=ngalert.state.manager t=2024-04-14T15:11:35.821711711Z level=info msg="State cache has been initialized" states=0 duration=148.708µs 2024-04-14T15:11:36.106 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=ticker t=2024-04-14T15:11:35.821833318Z level=info msg=starting first_tick=2024-04-14T15:11:40Z 2024-04-14T15:11:36.107 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=ngalert.multiorg.alertmanager t=2024-04-14T15:11:35.821875298Z level=info msg="starting MultiOrg Alertmanager" 2024-04-14T15:11:36.107 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=http.server t=2024-04-14T15:11:35.823528546Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-04-14T15:11:36.107 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:11:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore.transactions t=2024-04-14T15:11:35.845642625Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-04-14T15:11:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:36 smithi071 ceph-mon[33150]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:36 smithi071 ceph-mon[33150]: Deploying daemon node-exporter.a on smithi045 2024-04-14T15:11:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:36 smithi045 ceph-mon[37795]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:36 smithi045 ceph-mon[37795]: Deploying daemon node-exporter.a on smithi045 2024-04-14T15:11:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:36 smithi045 ceph-mon[29624]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:36 smithi045 ceph-mon[29624]: Deploying daemon node-exporter.a on smithi045 2024-04-14T15:11:36.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:36 smithi045 bash[72858]: Getting image source signatures 2024-04-14T15:11:36.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:36 smithi045 bash[72858]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-04-14T15:11:36.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:36 smithi045 bash[72858]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-04-14T15:11:36.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:36 smithi045 bash[72858]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-04-14T15:11:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:11:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:11:37.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:37 smithi045 bash[72858]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-04-14T15:11:37.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:37 smithi045 bash[72858]: Writing manifest to image destination 2024-04-14T15:11:38.041 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:37 smithi045 podman[72858]: 2024-04-14 15:11:37.730579741 +0000 UTC m=+2.196409719 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-04-14T15:11:38.041 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:37 smithi045 podman[72858]: 2024-04-14 15:11:37.839789722 +0000 UTC m=+2.305619700 container create f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-mon[29624]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-mon[37795]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 podman[72858]: 2024-04-14 15:11:38.04189103 +0000 UTC m=+2.507721009 container init f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 podman[72858]: 2024-04-14 15:11:38.044891742 +0000 UTC m=+2.510721729 container start f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.049Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.049Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-14T15:11:38.497 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=arp 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-14T15:11:38.498 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=edac 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-14T15:11:38.499 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.050Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=os 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-14T15:11:38.500 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=stat 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=time 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=uname 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a[72910]: ts=2024-04-14T15:11:38.051Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 bash[72858]: f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 2024-04-14T15:11:38.501 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:11:38 smithi045 systemd[1]: Started Ceph node-exporter.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:38 smithi071 ceph-mon[33150]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:38 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[37795]: Deploying daemon node-exporter.b on smithi071 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:39 smithi045 ceph-mon[29624]: Deploying daemon node-exporter.b on smithi071 2024-04-14T15:11:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:39 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:39 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:39 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:39.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:39 smithi071 ceph-mon[33150]: Deploying daemon node-exporter.b on smithi071 2024-04-14T15:11:39.608 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:39 smithi071 bash[64104]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-04-14T15:11:40.206 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:39 smithi071 bash[64104]: Getting image source signatures 2024-04-14T15:11:40.206 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:39 smithi071 bash[64104]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-04-14T15:11:40.206 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:39 smithi071 bash[64104]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-04-14T15:11:40.206 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:39 smithi071 bash[64104]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-04-14T15:11:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:40 smithi045 ceph-mon[37795]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:40 smithi045 ceph-mon[29624]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:40.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:40 smithi071 ceph-mon[33150]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:41.607 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:41 smithi071 bash[64104]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-04-14T15:11:41.669 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:41 smithi071 bash[64104]: Writing manifest to image destination 2024-04-14T15:11:42.101 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:41 smithi071 podman[64104]: 2024-04-14 15:11:41.716940668 +0000 UTC m=+2.449660692 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-04-14T15:11:42.101 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:41 smithi071 podman[64104]: 2024-04-14 15:11:41.824056796 +0000 UTC m=+2.556776814 container create 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:11:42.101 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 podman[64104]: 2024-04-14 15:11:42.026241681 +0000 UTC m=+2.758961702 container init 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:11:42.101 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 podman[64104]: 2024-04-14 15:11:42.02898195 +0000 UTC m=+2.761701970 container start 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.033Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=arp 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=bcache 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=bonding 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=btrfs 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=conntrack 2024-04-14T15:11:42.102 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=cpu 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=diskstats 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=dmi 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=edac 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=entropy 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=filefd 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=filesystem 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=hwmon 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=infiniband 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=ipvs 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=loadavg 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=mdadm 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=meminfo 2024-04-14T15:11:42.103 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=netclass 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=netdev 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=netstat 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=nfs 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=nfsd 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=nvme 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=os 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=pressure 2024-04-14T15:11:42.104 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=rapl 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=schedstat 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=selinux 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=sockstat 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=softnet 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=stat 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=tapestats 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=textfile 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=time 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=uname 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=vmstat 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=xfs 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.034Z caller=node_exporter.go:117 level=info collector=zfs 2024-04-14T15:11:42.105 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.035Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-04-14T15:11:42.106 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b[64154]: ts=2024-04-14T15:11:42.035Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-04-14T15:11:42.106 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 bash[64104]: 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e 2024-04-14T15:11:42.357 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:11:42 smithi071 systemd[1]: Started Ceph node-exporter.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:42 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:42 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]: dispatch 2024-04-14T15:11:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-04-14T15:11:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:43.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[37795]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-04-14T15:11:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[37795]: osdmap e71: 8 total, 8 up, 8 in 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[29624]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-04-14T15:11:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:44 smithi045 ceph-mon[29624]: osdmap e71: 8 total, 8 up, 8 in 2024-04-14T15:11:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:44 smithi071 ceph-mon[33150]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:11:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 5]}]': finished 2024-04-14T15:11:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-04-14T15:11:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-04-14T15:11:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:44 smithi071 ceph-mon[33150]: osdmap e71: 8 total, 8 up, 8 in 2024-04-14T15:11:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[29624]: osdmap e72: 8 total, 8 up, 8 in 2024-04-14T15:11:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[29624]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[37795]: osdmap e72: 8 total, 8 up, 8 in 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[37795]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:45 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:45 smithi071 ceph-mon[33150]: osdmap e72: 8 total, 8 up, 8 in 2024-04-14T15:11:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:45 smithi071 ceph-mon[33150]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:45 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:45 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:46 smithi071 ceph-mon[33150]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-14T15:11:46.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:46 smithi045 ceph-mon[37795]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-14T15:11:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:46 smithi045 ceph-mon[29624]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-14T15:11:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:11:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:11:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: Reconfiguring alertmanager.a (dependencies changed)... 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[37795]: Reconfiguring daemon alertmanager.a on smithi045 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: Reconfiguring alertmanager.a (dependencies changed)... 2024-04-14T15:11:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:47 smithi045 ceph-mon[29624]: Reconfiguring daemon alertmanager.a on smithi045 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:11:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: Reconfiguring alertmanager.a (dependencies changed)... 2024-04-14T15:11:47.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:47 smithi071 ceph-mon[33150]: Reconfiguring daemon alertmanager.a on smithi045 2024-04-14T15:11:49.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:49 smithi045 ceph-mon[37795]: pgmap v38: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:11:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:49 smithi045 ceph-mon[29624]: pgmap v38: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:11:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:49 smithi071 ceph-mon[33150]: pgmap v38: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:11:50.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:50 smithi045 systemd[1]: Stopping Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:50 smithi071 ceph-mon[33150]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-14T15:11:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:50 smithi071 ceph-mon[33150]: Cluster is now healthy 2024-04-14T15:11:50.884 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:50 smithi045 ceph-mon[37795]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-14T15:11:50.884 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:50 smithi045 ceph-mon[37795]: Cluster is now healthy 2024-04-14T15:11:50.884 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:50 smithi045 ceph-mon[29624]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-14T15:11:50.884 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:50 smithi045 ceph-mon[29624]: Cluster is now healthy 2024-04-14T15:11:50.885 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[72070]: ts=2024-04-14T15:11:50.592Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-04-14T15:11:50.885 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:50 smithi045 podman[73621]: 2024-04-14 15:11:50.632997612 +0000 UTC m=+0.146508731 container died beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:51.246 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:50 smithi045 podman[73621]: 2024-04-14 15:11:50.983915505 +0000 UTC m=+0.497426630 container cleanup beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:51.246 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:50 smithi045 bash[73621]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a 2024-04-14T15:11:51.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:51 smithi045 ceph-mon[29624]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-04-14T15:11:51.561 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:51 smithi045 ceph-mon[37795]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-04-14T15:11:51.562 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 podman[73635]: 2024-04-14 15:11:51.310551512 +0000 UTC m=+0.679654633 container remove beef7887d36af0d83b1577d1f3d0ad4c9fd89f843bd249568792a918e84cf9cb (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:51.562 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 podman[73635]: 2024-04-14 15:11:51.411300132 +0000 UTC m=+0.780403213 volume remove 27cd2a3cb7c6cca96ffdf427c86eb0cdce59bfcefb99522691f32b8fadbf661a 2024-04-14T15:11:51.562 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-04-14T15:11:51.562 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 systemd[1]: Stopped Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:51.562 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 systemd[1]: Starting Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:51.853 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 podman[73726]: 2024-04-14 15:11:51.770518183 +0000 UTC m=+0.113291291 volume create 8a1a04bc4e9e519294b6465f812ca4b1029ba482b87c8b113aadd3ae120ba8d2 2024-04-14T15:11:51.853 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 podman[73726]: 2024-04-14 15:11:51.674981584 +0000 UTC m=+0.017754693 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-04-14T15:11:51.854 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:51 smithi045 podman[73726]: 2024-04-14 15:11:51.854201601 +0000 UTC m=+0.196974710 container create 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:51.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:51 smithi071 ceph-mon[33150]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-04-14T15:11:52.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 podman[73726]: 2024-04-14 15:11:52.190588618 +0000 UTC m=+0.533361727 container init 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 podman[73726]: 2024-04-14 15:11:52.193431294 +0000 UTC m=+0.536204407 container start 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.206Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.206Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.208Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.45 port=9094 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.209Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.233Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.233Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.236Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:52.236Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 bash[73726]: 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a 2024-04-14T15:11:52.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:52 smithi045 systemd[1]: Started Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:53.479 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:11:53.479 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.479 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.480 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-mon[33150]: Reconfiguring prometheus.a (dependencies changed)... 2024-04-14T15:11:53.480 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-mon[33150]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-04-14T15:11:53.480 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:53 smithi071 ceph-mon[33150]: Reconfiguring daemon prometheus.a on smithi071 2024-04-14T15:11:53.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[37795]: Reconfiguring prometheus.a (dependencies changed)... 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[37795]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[37795]: Reconfiguring daemon prometheus.a on smithi071 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[29624]: Reconfiguring prometheus.a (dependencies changed)... 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[29624]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-04-14T15:11:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:53 smithi045 ceph-mon[29624]: Reconfiguring daemon prometheus.a on smithi071 2024-04-14T15:11:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:54.497 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:11:54 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:11:54.210Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000930758s 2024-04-14T15:11:54.603 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:11:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:55 smithi071 ceph-mon[33150]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-04-14T15:11:55.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:55 smithi045 ceph-mon[37795]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-04-14T15:11:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:55 smithi045 ceph-mon[29624]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s; 32 B/s, 1 objects/s recovering 2024-04-14T15:11:56.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:55 smithi071 systemd[1]: Stopping Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:56.458 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[61814]: ts=2024-04-14T15:11:56.208Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-14T15:11:56.459 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 podman[65077]: 2024-04-14 15:11:56.218280819 +0000 UTC m=+0.117405559 container died 3c7d7117f239f521fd96dbb46007788eb6f433567109d77b8b7048ea8d362179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:56.750 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 podman[65077]: 2024-04-14 15:11:56.55803211 +0000 UTC m=+0.457156848 container cleanup 3c7d7117f239f521fd96dbb46007788eb6f433567109d77b8b7048ea8d362179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:56.750 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 bash[65077]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a 2024-04-14T15:11:57.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 podman[65091]: 2024-04-14 15:11:56.750896393 +0000 UTC m=+0.535222118 container remove 3c7d7117f239f521fd96dbb46007788eb6f433567109d77b8b7048ea8d362179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:57.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-04-14T15:11:57.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 systemd[1]: Stopped Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:11:57.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@prometheus.a.service: Consumed 1.083s CPU time. 2024-04-14T15:11:57.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:56 smithi071 systemd[1]: Starting Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:11:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:57 smithi071 ceph-mon[33150]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:11:57.607 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:57 smithi071 podman[65186]: 2024-04-14 15:11:57.098487479 +0000 UTC m=+0.019183215 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-04-14T15:11:57.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:57 smithi045 ceph-mon[37795]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:11:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:57 smithi045 ceph-mon[29624]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:11:58.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:11:58.858 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:58 smithi071 podman[65186]: 2024-04-14 15:11:58.398202144 +0000 UTC m=+1.318897862 container create 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:11:59.707 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-mon[33150]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:11:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-mon[37795]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:11:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-mon[29624]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 podman[65186]: 2024-04-14 15:11:59.708025832 +0000 UTC m=+2.628721552 container init 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 podman[65186]: 2024-04-14 15:11:59.711436005 +0000 UTC m=+2.632131726 container start 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.740Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.740Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.740Z caller=main.go:570 level=info host_details="(Linux 5.14.0-435.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Mar 29 18:46:37 UTC 2024 x86_64 smithi071 (none))" 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.740Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-04-14T15:12:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.740Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.742Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.743Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.744Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.744Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.747Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.747Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.119µs 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.747Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-04-14T15:12:00.109 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.748Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.748Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.748Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.466µs wal_replay_duration=666.16µs wbl_replay_duration=131ns total_replay_duration=709.506µs 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.749Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.749Z caller=main.go:1029 level=info msg="TSDB started" 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.749Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.769Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.138309ms db_storage=1.317µs remote_storage=1.099µs web_handler=503ns query_engine=669ns scrape=699.572µs scrape_sd=103.223µs notify=10.227µs notify_sd=9.089µs rules=18.997485ms tracing=6.009µs 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.769Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-04-14T15:12:00.110 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:11:59.769Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-04-14T15:12:00.111 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 bash[65186]: 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 2024-04-14T15:12:00.111 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:11:59 smithi071 systemd[1]: Started Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:12:00.240 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:59] ENGINE Bus STOPPING 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:59] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:59] ENGINE Bus STOPPED 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:11:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:11:59] ENGINE Bus STARTING 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Serving on http://:::9283 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STARTED 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STOPPING 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STOPPED 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STARTING 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Serving on http://:::9283 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STARTED 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STOPPING 2024-04-14T15:12:00.241 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:12:00.242 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STOPPED 2024-04-14T15:12:00.242 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STARTING 2024-04-14T15:12:00.242 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Serving on http://:::9283 2024-04-14T15:12:00.242 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:12:00] ENGINE Bus STARTED 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:00 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi045.front.sepia.ceph.com:9093"}]: dispatch 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi071.front.sepia.ceph.com:9095"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi071.front.sepia.ceph.com:3000"}]: dispatch 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:01.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:00 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:12:02.212 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:01 smithi045 ceph-mon[37795]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:12:02.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:01 smithi045 ceph-mon[29624]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:12:02.256 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:01 smithi071 ceph-mon[33150]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-04-14T15:12:02.496 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:12:02 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:12:02.212Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003548037s 2024-04-14T15:12:03.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:12:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:04 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:12:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:04 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:06 smithi045 ceph-mon[37795]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:06 smithi045 ceph-mon[29624]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:06 smithi071 ceph-mon[33150]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:08 smithi045 ceph-mon[37795]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:08 smithi045 ceph-mon[29624]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:08 smithi071 ceph-mon[33150]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:10 smithi045 ceph-mon[37795]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:10 smithi045 ceph-mon[29624]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:10 smithi071 ceph-mon[33150]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:12 smithi045 ceph-mon[37795]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:12.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:12 smithi045 ceph-mon[29624]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:12 smithi071 ceph-mon[33150]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:13.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:13.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:13.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:14.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:14 smithi045 ceph-mon[29624]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:14.460 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:14 smithi045 ceph-mon[37795]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:14 smithi071 ceph-mon[33150]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:16 smithi045 ceph-mon[37795]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:16 smithi045 ceph-mon[29624]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:16 smithi071 ceph-mon[33150]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:18 smithi045 ceph-mon[29624]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:18 smithi045 ceph-mon[37795]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:18 smithi071 ceph-mon[33150]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:20.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:20 smithi045 ceph-mon[29624]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:20 smithi045 ceph-mon[37795]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:20 smithi071 ceph-mon[33150]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:22.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:22 smithi071 ceph-mon[33150]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:22 smithi045 ceph-mon[37795]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:22 smithi045 ceph-mon[29624]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:23.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:24 smithi045 ceph-mon[37795]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:24 smithi045 ceph-mon[29624]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:24.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:24 smithi071 ceph-mon[33150]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:26 smithi045 ceph-mon[29624]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:26 smithi045 ceph-mon[37795]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:26 smithi071 ceph-mon[33150]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:28 smithi045 ceph-mon[29624]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:28 smithi045 ceph-mon[37795]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:28.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:28 smithi071 ceph-mon[33150]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:30 smithi045 ceph-mon[37795]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:30 smithi045 ceph-mon[29624]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:30 smithi071 ceph-mon[33150]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:32 smithi045 ceph-mon[37795]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:32 smithi045 ceph-mon[29624]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:32 smithi071 ceph-mon[33150]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:33.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:34 smithi045 ceph-mon[29624]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:34 smithi045 ceph-mon[37795]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:34 smithi071 ceph-mon[33150]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:36 smithi045 ceph-mon[29624]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:36 smithi045 ceph-mon[37795]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:36 smithi071 ceph-mon[33150]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:38 smithi045 ceph-mon[29624]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:38.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:38 smithi045 ceph-mon[37795]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:38 smithi071 ceph-mon[33150]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:40 smithi045 ceph-mon[29624]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:40 smithi045 ceph-mon[37795]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:40 smithi071 ceph-mon[33150]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:41.597 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 98% (12262/12473) Updating files: 99% (12349/12473) Updating files: 100% (12473/12473) Updating files: 100% (12473/12473), done. 2024-04-14T15:12:42.166 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '05c9d46e7de5db2cb434c177a6bbfc56f472d09b'. 2024-04-14T15:12:42.166 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.166 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-14T15:12:42.166 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-14T15:12:42.167 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 05c9d46e7de Merge pull request #56637 from adk3798/node-proxy-service-expected-daemon-count 2024-04-14T15:12:42.169 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-14T15:12:42.196 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-14T15:12:42.198 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-14T15:12:42.198 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-14T15:12:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:42 smithi045 ceph-mon[29624]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:42 smithi045 ceph-mon[37795]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:42 smithi071 ceph-mon[33150]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:12:43.019 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-14T15:12:43.091 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-14T15:12:43.132 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-14T15:12:43.133 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-14T15:12:43.133 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-14T15:12:43.172 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-14T15:12:43.175 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-14T15:12:43.175 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-14T15:12:43.230 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-04-14T15:12:43.231 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-04-14T15:12:43.232 DEBUG:teuthology.orchestra.run.smithi045:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-04-14T15:12:43.377 INFO:tasks.workunit.client.0.smithi045.stderr:+ parallel=1 2024-04-14T15:12:43.377 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' '' = --serial ']' 2024-04-14T15:12:43.377 INFO:tasks.workunit.client.0.smithi045.stderr:+ crimson=0 2024-04-14T15:12:43.377 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' '' = --crimson ']' 2024-04-14T15:12:43.378 INFO:tasks.workunit.client.0.smithi045.stderr:+ color= 2024-04-14T15:12:43.378 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -t 1 ']' 2024-04-14T15:12:43.378 INFO:tasks.workunit.client.0.smithi045.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-04-14T15:12:43.378 INFO:tasks.workunit.client.0.smithi045.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-14T15:12:43.378 INFO:tasks.workunit.client.0.smithi045.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:+ declare -A pids 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_aio 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_aio' 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_aio 2024-04-14T15:12:43.379 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.380 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_aio 2024-04-14T15:12:43.380 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74403 2024-04-14T15:12:43.380 INFO:tasks.workunit.client.0.smithi045.stdout:test api_aio on pid 74403 2024-04-14T15:12:43.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_aio on pid 74403' 2024-04-14T15:12:43.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74403 2024-04-14T15:12:43.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_aio_pp 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_aio_pp' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_aio_pp 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.384 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.385 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_aio_pp 2024-04-14T15:12:43.386 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74410 2024-04-14T15:12:43.387 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_aio_pp on pid 74410' 2024-04-14T15:12:43.387 INFO:tasks.workunit.client.0.smithi045.stdout:test api_aio_pp on pid 74410 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74410 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_io 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_io' 2024-04-14T15:12:43.391 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_io 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.393 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.394 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.395 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_io 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74419 2024-04-14T15:12:43.396 INFO:tasks.workunit.client.0.smithi045.stdout:test api_io on pid 74419 2024-04-14T15:12:43.401 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_io on pid 74419' 2024-04-14T15:12:43.401 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74419 2024-04-14T15:12:43.401 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.401 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-04-14T15:12:43.401 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_io_pp 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_io_pp' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_io_pp 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.402 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.403 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.404 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_io_pp 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74431 2024-04-14T15:12:43.405 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_io_pp on pid 74431' 2024-04-14T15:12:43.406 INFO:tasks.workunit.client.0.smithi045.stdout:test api_io_pp on pid 74431 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74431 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_asio 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_asio' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_asio 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.411 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.412 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.413 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.414 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.415 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.416 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_asio 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74444 2024-04-14T15:12:43.417 INFO:tasks.workunit.client.0.smithi045.stdout:test api_asio on pid 74444 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_asio on pid 74444' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74444 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_list 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_list' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_list 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.424 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.425 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.426 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.427 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.428 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.429 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74403/exe 2024-04-14T15:12:43.430 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_list 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74461 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_list on pid 74461' 2024-04-14T15:12:43.431 INFO:tasks.workunit.client.0.smithi045.stdout:test api_list on pid 74461 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74461 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_lock 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_lock' 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_lock 2024-04-14T15:12:43.436 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.437 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.438 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_aio.log 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.439 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_aio: /' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.440 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.441 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.442 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74410/exe 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.443 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.444 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.444 INFO:tasks.workunit.client.0.smithi045.stdout:test api_lock on pid 74481 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_lock 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74481 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_lock on pid 74481' 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74481 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_lock_pp 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.446 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_lock_pp' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.447 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_lock_pp 2024-04-14T15:12:43.448 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_aio_pp: /' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.449 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.450 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.451 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.452 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.453 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.454 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.454 INFO:tasks.workunit.client.0.smithi045.stdout:test api_lock_pp on pid 74500 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_lock_pp 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74500 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_lock_pp on pid 74500' 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74500 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74419/exe 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-04-14T15:12:43.457 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_misc 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.458 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_misc' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.459 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.460 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.461 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.462 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.463 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_io: /' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.464 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_io.log 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_misc 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74431/exe 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.465 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.466 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.467 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.468 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_io_pp: /' 2024-04-14T15:12:43.469 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74444/exe 2024-04-14T15:12:43.470 INFO:tasks.workunit.client.0.smithi045.stdout:test api_misc on pid 74531 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_misc 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74531 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_misc on pid 74531' 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74531 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.473 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_misc_pp 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.474 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_misc_pp' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.475 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_asio: /' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_misc_pp 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.476 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_asio.log 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.477 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.478 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.479 INFO:tasks.workunit.client.0.smithi045.stdout:test api_misc_pp on pid 74547 2024-04-14T15:12:43.480 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:43.484 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_misc_pp 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74547 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_misc_pp on pid 74547' 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74547 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.485 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-04-14T15:12:43.486 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.487 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.488 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.489 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_tier_pp 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.490 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_tier_pp' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.491 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_tier_pp 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.492 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.493 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.494 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.495 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.496 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.497 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.498 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.499 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.499 INFO:tasks.workunit.client.0.smithi045.stdout:test api_tier_pp on pid 74575 2024-04-14T15:12:43.501 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_tier_pp 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.502 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74575 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_tier_pp on pid 74575' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74575 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.503 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_pool 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_pool' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74500/exe 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.504 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_pool 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74481/exe 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.505 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.506 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.507 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74461/exe 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.508 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_lock_pp: /' 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.509 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_lock.log 2024-04-14T15:12:43.510 INFO:tasks.workunit.client.0.smithi045.stdout:test api_pool on pid 74602 2024-04-14T15:12:43.512 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_pool 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74602 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_pool on pid 74602' 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74602 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_list: /' 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_lock: /' 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_list.log 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.513 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.514 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_snapshots 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_snapshots' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.515 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.516 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.517 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_snapshots 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.518 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74531/exe 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.519 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.520 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_snapshots 2024-04-14T15:12:43.521 INFO:tasks.workunit.client.0.smithi045.stdout:test api_snapshots on pid 74649 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74649 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_snapshots on pid 74649' 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74649 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-04-14T15:12:43.525 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.526 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_misc.log 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_misc: /' 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.527 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_snapshots_pp 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_snapshots_pp' 2024-04-14T15:12:43.528 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.529 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.530 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74547/exe 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_snapshots_pp 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.531 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.532 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.533 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_snapshots_pp 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74703 2024-04-14T15:12:43.534 INFO:tasks.workunit.client.0.smithi045.stdout:test api_snapshots_pp on pid 74703 2024-04-14T15:12:43.536 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_snapshots_pp on pid 74703' 2024-04-14T15:12:43.536 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74703 2024-04-14T15:12:43.536 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.536 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.536 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_misc_pp: /' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_stat 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_stat' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.537 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.538 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.539 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_stat 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.540 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_stat 2024-04-14T15:12:43.541 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.541 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.541 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74733 2024-04-14T15:12:43.541 INFO:tasks.workunit.client.0.smithi045.stdout:test api_stat on pid 74733 2024-04-14T15:12:43.545 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_stat on pid 74733' 2024-04-14T15:12:43.545 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74733 2024-04-14T15:12:43.545 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.546 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.547 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.548 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.549 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_stat_pp 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_stat_pp' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.550 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.551 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.552 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.553 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74575/exe 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_stat_pp 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.554 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_tier_pp: /' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.555 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.556 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.557 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.558 INFO:tasks.workunit.client.0.smithi045.stdout:test api_stat_pp on pid 74764 2024-04-14T15:12:43.561 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_stat_pp 2024-04-14T15:12:43.561 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74602/exe 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74649/exe 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74764 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_stat_pp on pid 74764' 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74764 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.562 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_watch_notify 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.563 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_pool: /' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_watch_notify' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_snapshots: /' 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_watch_notify 2024-04-14T15:12:43.564 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.565 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.566 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_pool.log 2024-04-14T15:12:43.567 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.567 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:43.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.568 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.569 INFO:tasks.workunit.client.0.smithi045.stdout:test api_watch_notify on pid 74810 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_watch_notify 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74810 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_watch_notify on pid 74810' 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74810 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_watch_notify_pp 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_watch_notify_pp' 2024-04-14T15:12:43.573 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.574 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.575 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_watch_notify_pp 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.576 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.577 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.578 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.579 INFO:tasks.workunit.client.0.smithi045.stdout:test api_watch_notify_pp on pid 74836 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_watch_notify_pp 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74836 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_watch_notify_pp on pid 74836' 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74836 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.585 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.586 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.587 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.588 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.589 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_cmd 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_cmd' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.590 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74703/exe 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.591 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74733/exe 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.592 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.593 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.594 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_cmd 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.595 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_stat.log 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.596 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_cmd 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_stat: /' 2024-04-14T15:12:43.597 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74911 2024-04-14T15:12:43.598 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_cmd on pid 74911' 2024-04-14T15:12:43.598 INFO:tasks.workunit.client.0.smithi045.stdout:test api_cmd on pid 74911 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74911 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_cmd_pp 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_cmd_pp' 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.603 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.604 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.605 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.606 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74764/exe 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.607 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.608 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.609 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.610 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_stat_pp: /' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_cmd_pp 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.611 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.612 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74810/exe 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.613 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_watch_notify: /' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.614 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.615 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.616 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.617 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.618 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74911/exe 2024-04-14T15:12:43.619 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.619 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-04-14T15:12:43.619 INFO:tasks.workunit.client.0.smithi045.stdout:test api_cmd_pp on pid 74995 2024-04-14T15:12:43.621 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_cmd_pp 2024-04-14T15:12:43.621 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74995 2024-04-14T15:12:43.621 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_cmd_pp on pid 74995' 2024-04-14T15:12:43.621 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=74995 2024-04-14T15:12:43.621 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.622 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_service 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_service' 2024-04-14T15:12:43.623 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_cmd.log 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.624 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74836/exe 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_cmd: /' 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-04-14T15:12:43.625 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-04-14T15:12:43.626 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_service 2024-04-14T15:12:43.626 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.626 INFO:tasks.workunit.client.0.smithi045.stdout:test api_service on pid 75025 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_service 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.628 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75025 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_service on pid 75025' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75025 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.629 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.630 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_service_pp 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_service_pp' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_service_pp 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.631 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.632 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.632 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.632 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.632 INFO:tasks.workunit.client.0.smithi045.stdout:test api_service_pp on pid 75053 2024-04-14T15:12:43.634 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.634 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.634 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.634 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_service_pp 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75053 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_service_pp on pid 75053' 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75053 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.635 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_c_write_operations 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_c_write_operations' 2024-04-14T15:12:43.636 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.637 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_c_write_operations 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.638 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.639 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.639 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.639 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.639 INFO:tasks.workunit.client.0.smithi045.stdout:test api_c_write_operations on pid 75065 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_c_write_operations 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75065 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_c_write_operations on pid 75065' 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75065 2024-04-14T15:12:43.643 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.644 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_c_read_operations 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_c_read_operations' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.645 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.646 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_c_read_operations 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.647 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.648 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.649 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.650 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.651 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_c_read_operations 2024-04-14T15:12:43.654 INFO:tasks.workunit.client.0.smithi045.stdout:test api_c_read_operations on pid 75115 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75115 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_c_read_operations on pid 75115' 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75115 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.658 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.659 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.660 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.661 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.662 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s api_cls_remote_reads 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' api_cls_remote_reads' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo api_cls_remote_reads 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75025/exe 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.663 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.664 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/74995/exe 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=api_cls_remote_reads 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.665 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.666 INFO:tasks.workunit.client.0.smithi045.stdout:test api_cls_remote_reads on pid 75138 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75138 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test api_cls_remote_reads on pid 75138' 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75138 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.670 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s list_parallel 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' list_parallel' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_service.log 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo list_parallel 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_service: /' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.671 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.672 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.673 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.674 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.675 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.676 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.677 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.678 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=list_parallel 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.679 INFO:tasks.workunit.client.0.smithi045.stdout:test list_parallel on pid 75177 2024-04-14T15:12:43.684 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75177 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test list_parallel on pid 75177' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75177 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.685 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s open_pools_parallel 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' open_pools_parallel' 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo open_pools_parallel 2024-04-14T15:12:43.686 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75065/exe 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.687 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.688 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.689 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.690 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.691 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.692 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.693 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75053/exe 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.694 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=open_pools_parallel 2024-04-14T15:12:43.695 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.695 INFO:tasks.workunit.client.0.smithi045.stdout:test open_pools_parallel on pid 75227 2024-04-14T15:12:43.697 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75227 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test open_pools_parallel on pid 75227' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75227 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s delete_pools_parallel 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' delete_pools_parallel' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.698 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.699 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo delete_pools_parallel 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.700 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.701 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_service_pp: /' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.702 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75115/exe 2024-04-14T15:12:43.703 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.704 INFO:tasks.workunit.client.0.smithi045.stdout:test delete_pools_parallel on pid 75246 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=delete_pools_parallel 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75246 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test delete_pools_parallel on pid 75246' 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75246 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.709 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.710 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.711 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-04-14T15:12:43.712 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s cls 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.713 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' cls' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75138/exe 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.714 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.715 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-04-14T15:12:43.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo cls 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.717 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.718 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.719 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.720 INFO:tasks.workunit.client.0.smithi045.stdout:test cls on pid 75272 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=cls 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75272 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test cls on pid 75272' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75272 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s cmd 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' cmd' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.727 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.728 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.729 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.730 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.731 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.732 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.733 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75246/exe 2024-04-14T15:12:43.734 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo cmd 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.735 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.736 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.737 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.738 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=cmd 2024-04-14T15:12:43.739 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75327 2024-04-14T15:12:43.740 INFO:tasks.workunit.client.0.smithi045.stdout:test cmd on pid 75327 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test cmd on pid 75327' 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75327 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75177/exe 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.744 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.745 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.746 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_list_parallel.log 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s handler_error 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' handler_error' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.747 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.748 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo handler_error 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ list_parallel: /' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.749 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.750 INFO:tasks.workunit.client.0.smithi045.stdout:test handler_error on pid 75402 2024-04-14T15:12:43.754 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=handler_error 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75402 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test handler_error on pid 75402' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75402 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.755 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.756 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.757 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.758 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.759 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.760 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.761 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75272/exe 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.762 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s io 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' io' 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.763 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ cls: /' 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_cls 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75227/exe 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.764 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_cls.log 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.765 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-04-14T15:12:43.766 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo io 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75327/exe 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.767 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.768 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_cmd.log 2024-04-14T15:12:43.769 INFO:tasks.workunit.client.0.smithi045.stdout:test io on pid 75506 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=io 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75506 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test io on pid 75506' 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75506 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s ec_io 2024-04-14T15:12:43.771 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' ec_io' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.772 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.773 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ cmd: /' 2024-04-14T15:12:43.773 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo ec_io 2024-04-14T15:12:43.773 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_cmd 2024-04-14T15:12:43.773 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.773 INFO:tasks.workunit.client.0.smithi045.stdout:test ec_io on pid 75542 2024-04-14T15:12:43.776 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=ec_io 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75542 2024-04-14T15:12:43.777 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test ec_io on pid 75542' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75542 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.778 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s list 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' list' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.779 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.780 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.781 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75402/exe 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo list 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.782 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.783 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.784 INFO:tasks.workunit.client.0.smithi045.stdout:test list on pid 75588 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=list 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75588 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test list on pid 75588' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.786 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75588 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.787 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s ec_list 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' ec_list' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_handler_error 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_handler_error.log 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.788 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.789 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo ec_list 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ handler_error: /' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.790 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.791 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.792 INFO:tasks.workunit.client.0.smithi045.stdout:test ec_list on pid 75620 2024-04-14T15:12:43.796 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=ec_list 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75620 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test ec_list on pid 75620' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75620 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.797 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.798 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.799 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.800 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.801 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.802 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s misc 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' misc' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo misc 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.803 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.804 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.805 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.806 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.807 INFO:tasks.workunit.client.0.smithi045.stdout:test misc on pid 75684 2024-04-14T15:12:43.811 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=misc 2024-04-14T15:12:43.811 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.811 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75684 2024-04-14T15:12:43.811 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test misc on pid 75684' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75684 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75506/exe 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.812 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s pool 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.813 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.814 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.815 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' pool' 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.816 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.817 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_io 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_io.log 2024-04-14T15:12:43.818 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75542/exe 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo pool 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.819 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ io: /' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ ec_io: /' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75588/exe 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.820 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_ec_io 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.821 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_ec_io.log 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.822 INFO:tasks.workunit.client.0.smithi045.stdout:test pool on pid 75738 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=pool 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75738 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test pool on pid 75738' 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75738 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ list: /' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s read_operations 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' read_operations' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.826 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_list 2024-04-14T15:12:43.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_list.log 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.828 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.829 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.830 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.831 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.831 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo read_operations 2024-04-14T15:12:43.831 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75620/exe 2024-04-14T15:12:43.831 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.831 INFO:tasks.workunit.client.0.smithi045.stdout:test read_operations on pid 75764 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=read_operations 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75764 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test read_operations on pid 75764' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75764 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.835 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s snapshots 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' snapshots' 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.836 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ ec_list: /' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_ec_list.log 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_ec_list 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.837 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.838 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo snapshots 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.839 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.840 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.841 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.842 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.842 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=snapshots 2024-04-14T15:12:43.842 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75786 2024-04-14T15:12:43.842 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test snapshots on pid 75786' 2024-04-14T15:12:43.842 INFO:tasks.workunit.client.0.smithi045.stdout:test snapshots on pid 75786 2024-04-14T15:12:43.845 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75786 2024-04-14T15:12:43.845 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.845 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.845 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.845 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.846 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s watch_notify 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' watch_notify' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo watch_notify 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.847 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.848 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.849 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75684/exe 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.850 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.852 INFO:tasks.workunit.client.0.smithi045.stdout:test watch_notify on pid 75801 2024-04-14T15:12:43.855 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.855 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=watch_notify 2024-04-14T15:12:43.855 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75801 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test watch_notify on pid 75801' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75801 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.856 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:++ printf %25s write_operations 2024-04-14T15:12:43.857 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ r=' write_operations' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_misc 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_misc.log 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.858 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ misc: /' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75738/exe 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ awk '{print $1}' 2024-04-14T15:12:43.859 INFO:tasks.workunit.client.0.smithi045.stderr:++ echo write_operations 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_pool 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.860 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.861 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.862 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.862 INFO:tasks.workunit.client.0.smithi045.stderr:+ ff=write_operations 2024-04-14T15:12:43.862 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75829 2024-04-14T15:12:43.862 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'test write_operations on pid 75829' 2024-04-14T15:12:43.862 INFO:tasks.workunit.client.0.smithi045.stdout:test write_operations on pid 75829 2024-04-14T15:12:43.864 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ pids[$f]=75829 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ ret=0 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 1 -eq 1 ']' 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74575 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74575 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.874 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.875 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_pool.log 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ pool: /' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -f /etc/bashrc ']' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+ . /etc/bashrc 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -z '' ']' 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ BASHRCSOURCED=Y 2024-04-14T15:12:43.876 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ shopt -q login_shell 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:+++ umask 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' 0022 -eq 0 ']' 2024-04-14T15:12:43.877 INFO:tasks.workunit.client.0.smithi045.stderr:++ SHELL=/bin/bash 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/bash_completion.sh 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/cobbler.sh 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.878 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorgrep.sh 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'grep=grep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.879 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'egrep=egrep --color=auto' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorls.sh 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' '!' -t 0 ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:+++ return 2024-04-14T15:12:43.880 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export S_COLORS=auto 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:+++ S_COLORS=auto 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.881 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.882 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.883 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.884 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75764/exe 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-04-14T15:12:43.885 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ /usr/libexec/grepconf.sh -c 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.886 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.887 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_read_operations.log 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_read_operations 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ read_operations: /' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.888 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.889 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75786/exe 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.890 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/debuginfod.sh 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/gawk.sh 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/lang.sh 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.891 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.892 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/locale 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75829/exe 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_snapshots 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ snapshots: /' 2024-04-14T15:12:43.893 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_snapshots.log 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG_backup=en_US.UTF-8 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /etc/locale.conf ']' 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -x /usr/bin/sed ']' 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.894 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_write_operations 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_write_operations.log 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ write_operations: /' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:++++ export LANG=en_US.UTF-8 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:++++ LANG=en_US.UTF-8 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ LANG=en_US.UTF-8 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ unset LANG_backup config 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n '' ']' 2024-04-14T15:12:43.895 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' dumb = linux ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/less.sh 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ for i in /etc/profile.d/*.sh 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ '[' '' ']' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++ . /etc/profile.d/which2.sh 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++++ readlink /proc/75801/exe 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:++++ basename /usr/bin/bash 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_declare='declare -f' 2024-04-14T15:12:43.896 INFO:tasks.workunit.client.0.smithi045.stderr:+++ which_opt=-f 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export which_declare 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+++ export -f which 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset i 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:++ unset -f pathmunge 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ export PATH 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ unset rc 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph_test_neorados_watch_notify 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-04-14T15:12:43.897 INFO:tasks.workunit.client.0.smithi045.stderr:+ sed 's/^/ watch_notify: /' 2024-04-14T15:12:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1733241285' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1733241285' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:44 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1733241285' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:44 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: osdmap e73: 8 total, 8 up, 8 in 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1465755156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/724938047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/805397667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1991579260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4152370865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4211196195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3719828916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1248739655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3281295515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1059216615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/839582766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2398607577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2103452952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/427626099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4028490466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/630232505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[29624]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: osdmap e73: 8 total, 8 up, 8 in 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1465755156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/724938047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/805397667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1991579260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4152370865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4211196195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.502 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3719828916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1248739655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3281295515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1059216615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/839582766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2398607577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2103452952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/427626099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4028490466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/630232505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:45 smithi045 ceph-mon[37795]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi045-75731-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi045-75769-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: osdmap e73: 8 total, 8 up, 8 in 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1465755156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/724938047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/805397667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1991579260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4152370865' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4211196195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3719828916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1248739655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3281295515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.34868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1059216615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/839582766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2398607577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2103452952' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/427626099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4028490466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/630232505' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:45 smithi071 ceph-mon[33150]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:45.644 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-04-14T15:12:45.644 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [----------] Global test environment set-up. 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [----------] 12 tests from AsioRados 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (13 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (6 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-04-14T15:12:45.645 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [----------] 12 tests from AsioRados (34 ms total) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [----------] Global test environment tear-down 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (2186 ms total) 2024-04-14T15:12:45.646 INFO:tasks.workunit.client.0.smithi045.stdout: api_asio: [ PASSED ] 12 tests. 2024-04-14T15:12:45.977 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-04-14T15:12:45.977 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] Global test environment set-up. 2024-04-14T15:12:45.977 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 7 tests from LibRadosList 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.ListObjects (440 ms) 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (44 ms) 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo1,foo2,foo3 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo1 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo2 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo3 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo1,foo4,foo5 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo4 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo5 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo1 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo6,foo7 2024-04-14T15:12:45.978 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo7 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo6 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo4 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo5 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns2:foo7 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns2:foo6 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo1 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo1 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo2 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo3 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (91 ms) 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 1 0 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 10 0 2024-04-14T15:12:45.979 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 13 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 7 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 14 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 0 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 15 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 11 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 5 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 8 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 6 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 3 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 4 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 12 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 9 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 2 0 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (77 ms) 2024-04-14T15:12:45.980 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: x cursor=MIN 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-04-14T15:12:45.981 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: FIRST> seek to MIN oid=1 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:566253c9:::13:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-04-14T15:12:45.982 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:863748b0:::15:head 2024-04-14T15:12:46.042 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-04-14T15:12:46.043 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:e960b815:::9:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-04-14T15:12:46.044 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=2 expected=2 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:e960b815:::9:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=9 expected=9 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=6 expected=6 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=8 expected=8 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-04-14T15:12:46.045 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=5 expected=5 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=11 expected=11 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:863748b0:::15:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=15 expected=15 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-04-14T15:12:46.046 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=0 list_parallel: process_1_[75418]: starting. 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: creating pool ceph_test_rados_list_parallel.smithi045-75363 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: created object 0... 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: created object 25... 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: created object 49... 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: finishing. 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_1_[75418]: shutting down. 2024-04-14T15:12:46.192 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: starting. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: listing objects. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: listed object 0... 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: listed object 25... 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: saw 50 objects 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_2_[75419]: shutting down. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: starting. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: creating pool ceph_test_rados_list_parallel.smithi045-75363 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: created object 0... 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: created object 25... 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: created object 49... 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: finishing. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_3_[76146]: shutting down. 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.193 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: starting. 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: listing objects. 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: listed object 0... 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: listed object 25... 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: saw 44 objects 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_4_[76147]: shutting down. 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: starting. 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: removed 25 objects... 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: removed half of the objects 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: removed 50 objects... 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: removed 50 objects 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_5_[76148]: shutting down. 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.194 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: starting. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: creating pool ceph_test_rados_list_parallel.smithi045-75363 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: created object 0... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: created object 25... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: created object 49... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: finishing. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_6_[76194]: shutting down. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: starting. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: listing objects. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: listed object 0... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: listed object 25... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: listed object 50... 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: saw 56 objects 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_7_[76195]: shutting down. 2024-04-14T15:12:46.195 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: starting. 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: added 25 objects... 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: added half of the objects 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: added 50 objects... 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: added 50 objects 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_8_[76196]: shutting down. 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.196 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ********************** handler_error: Running main() from gmock_main.cc 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [----------] Global test environment set-up. 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (2465 ms) 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [----------] 1 test from neocls_handler_error (2465 ms total) 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: 2024-04-14T15:12:46.244 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [----------] Global test environment tear-down 2024-04-14T15:12:46.245 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [==========] 1 test from 1 test suite ran. (2465 ms total) 2024-04-14T15:12:46.245 INFO:tasks.workunit.client.0.smithi045.stdout: handler_error: [ PASSED ] 1 test. 2024-04-14T15:12:46.246 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (348 ms) 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (24 ms) 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (42 ms) 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (33 ms) 2024-04-14T15:12:46.247 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (7 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (7 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (5 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-04-14T15:12:46.248 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (15 ms) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (540 ms total) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (2591 ms total) 2024-04-14T15:12:46.249 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (2558 ms) 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (2558 ms total) 2024-04-14T15:12:46.268 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: 2024-04-14T15:12:46.269 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-04-14T15:12:46.269 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (2558 ms total) 2024-04-14T15:12:46.269 INFO:tasks.workunit.client.0.smithi045.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (55 ms) 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (55 ms) 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (2512 ms) 2024-04-14T15:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (2622 ms total) 2024-04-14T15:12:46.271 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: 2024-04-14T15:12:46.271 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-04-14T15:12:46.271 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (2622 ms total) 2024-04-14T15:12:46.271 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: Running main() from gmock_main.cc 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [----------] Global test environment set-up. 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (60 ms) 2024-04-14T15:12:46.388 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (41 ms) 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (2570 ms) 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.203224+0000 mon.a [WRN] Health check failed: 35 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214342+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214456+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214563+0000 mon.a [INF] from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214701+0000 mon.a [INF] from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214822+0000 mon.a [INF] from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.214922+0000 mon.a [INF] from='client.34694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215009+0000 mon.a [INF] from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215097+0000 mon.a [INF] from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.389 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215169+0000 mon.a [INF] from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.390 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215250+0000 mon.a [INF] from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.390 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215306+0000 mon.a [INF] from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: pgmap v67: 1252 pgs: 1120 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: Health check failed: 35 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: osdmap e74: 8 total, 8 up, 8 in 2024-04-14T15:12:46.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: pgmap v67: 1252 pgs: 1120 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: Health check failed: 35 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: osdmap e74: 8 total, 8 up, 8 in 2024-04-14T15:12:46.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: pgmap v67: 1252 pgs: 1120 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:12:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: Health check failed: 35 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1923041739' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1673444199' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi045-74490-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi045-74524-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi045-74511-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi045-74470-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi045-74591-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi045-74587-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi045-74598-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75263-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:46.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: osdmap e74: 8 total, 8 up, 8 in 2024-04-14T15:12:46.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:46.679 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215399+0000 mon.a [INF] from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi045-75251-1","app": "rados","yes_i_really_mean_it": true}]': fin********* 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: starting. 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: creating pool ceph_test_rados_list_parallel.smithi045-75363 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: created object 0... 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: created object 25... 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: created object 49... 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: finishing. 2024-04-14T15:12:46.680 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_9_[76242]: shutting down. 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: starting. 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: listing objects. 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: listed object 0... 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: listed object 25... 2024-04-14T15:12:46.681 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: listed object 50... 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: listed object 75... 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: saw 97 objects 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_10_[76243]: shutting down. 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: starting. 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: removed 25 objects... 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: removed half of the objects 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: removed 50 objects... 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: removed 50 objects 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_13_[76246]: shutting down. 2024-04-14T15:12:46.682 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: starting. 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: added 25 objects... 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: added half of the objects 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: added 50 objects... 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: added 50 objects 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_11_[76244]: shutting down. 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: starting. 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: added 25 objects... 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: added half of the objects 2024-04-14T15:12:46.683 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: added 50 objects... 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: added 50 objects 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_12_[76245]: shutting down. 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: starting. 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: creating pool ceph_test_rados_list_parallel.smithi045-75363 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: created object 0... 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: created object 25... 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: created object 49... 2024-04-14T15:12:46.684 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: finishing. 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_14_[76641]: shutting down. 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: starting. 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listing objects. 2024-04-14T15:12:46.685 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 0... 2024-04-14T15:12:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:47.248 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 25... 2024-04-14T15:12:47.248 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 50... 2024-04-14T15:12:47.248 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 75... 2024-04-14T15:12:47.248 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 100... 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: listed object 125... 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: saw 150 objects 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_15_[76642]: shutting down. 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: starting. 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: added 25 objects... 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: added half of the objects 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: added 50 objects... 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: added 50 objects 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: process_16_[76643]: shutting down. 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.249 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.250 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.250 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.250 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******************************* 2024-04-14T15:12:47.250 INFO:tasks.workunit.client.0.smithi045.stdout: list_parallel: ******* SUCCESS ********** 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout:ished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215491+0000 mon.a [INF] from='client.34868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75161-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215573+0000 mon.a [INF] from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi045-74876-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215658+0000 mon.a [INF] from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi045-74940-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215723+0000 mon.a [INF] from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi045-74975-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215818+0000 mon.a [INF] from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215880+0000 mon.a [INF] from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi045-74775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215940+0000 mon.a [INF] from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi045-74757-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:45.215999+0000 mon.a [INF] from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi045-74884-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:47.282 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.225405+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.225497+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.225595+0000 mon.a [INF] from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.240312+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.363189+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.365523+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.382771+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.385583+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.387016+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.387448+0000 client.admin [INF] onexx 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.399953+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.401900+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.417691+0000 mon.a [INF] from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.419806+0000 mon.a [INF] from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.422398+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.426096+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.460429+0000 mon.a [INF] from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.460796+0000 mon.a [INF] from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.460998+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.463103+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.465416+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.470379+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.470854+0000 mon.a [INF] from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.471591+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:47.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:47.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:47.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: osdmap e75: 8 total, 8 up, 8 in 2024-04-14T15:12:47.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: onexx 2024-04-14T15:12:47.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.612 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.613 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.613 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.613 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: osdmap e75: 8 total, 8 up, 8 in 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: onexx 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi045-75731-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:47.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi045-75020-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi045-75769-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: osdmap e75: 8 total, 8 up, 8 in 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: onexx 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:47.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: pgmap v70: 668 pgs: 536 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:12:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: osdmap e76: 8 total, 8 up, 8 in 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3583539346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3358422846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3159971822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: twoxx 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.611 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: pgmap v70: 668 pgs: 536 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:12:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: osdmap e76: 8 total, 8 up, 8 in 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3583539346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3358422846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: pgmap v70: 668 pgs: 536 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi045-74511-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: osdmap e76: 8 total, 8 up, 8 in 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3583539346' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3358422846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:48.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3159971822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: twoxx 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3159971822' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: twoxx 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.35114 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:12:48.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:12:48.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15: cls: Running main() from gmock_main.cc 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [----------] Global test environment set-up. 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [ OK ] NeoRadosCls.DNE (2507 ms) 2024-04-14T15:12:49.256 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (3015 ms) 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [----------] 2 tests from NeoRadosCls (5522 ms total) 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [----------] Global test environment tear-down 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [==========] 2 tests from 1 test suite ran. (5522 ms total) 2024-04-14T15:12:49.257 INFO:tasks.workunit.client.0.smithi045.stdout: cls: [ PASSED ] 2 tests. 2024-04-14T15:12:49.271 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: starting. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: creating pool ceph_test_rados_delete_pools_parallel.smithi045-75315 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: created object 0... 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: created object 25... 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: created object 49... 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: finishing. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_1_[75398]: shutting down. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_2_[75399]: starting. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_2_[75399]: deleting pool ceph_test_rados_delete_pools_parallel.smithi045-75315 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_2_[75399]: shutting down. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******************************* 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: starting. 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: creating pool ceph_test_rados_delete_pools_parallel.smithi045-75315 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: created object 0... 2024-04-14T15:12:49.272 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: created object 25... 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: created object 49... 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: finishing. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_3_[76327]: shutting down. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******************************* 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: starting. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: listing objects. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: listed object 0... 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: listed object 25... 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: saw 50 objects 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_5_[76329]: shutting down. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******************************* 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_4_[76328]: starting. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_4_[76328]: deleting pool ceph_test_rados_delete_pools_parallel.smithi045-75315 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: process_4_[76328]: shutting down. 2024-04-14T15:12:49.273 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******************************* 2024-04-14T15:12:49.274 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******************************* 2024-04-14T15:12:49.274 INFO:tasks.workunit.client.0.smithi045.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-04-14T15:12:49.277 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: starting. 2024-04-14T15:12:49.277 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: creating pool ceph_test_rados_open_pools_parallel.smithi045-75452 2024-04-14T15:12:49.277 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: created object 0... 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: created object 25... 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: created object 49... 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: finishing. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_1_[75539]: shutting down. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_2_[75540]: starting. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_2_[75540]: rados_pool_create. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_2_[75540]: rados_ioctx_create. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_2_[75540]: shutting down. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: ******************************* 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: starting. 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: creating pool ceph_test_rados_open_pools_parallel.smithi045-75452 2024-04-14T15:12:49.278 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: created object 0... 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: created object 25... 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: created object 49... 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: finishing. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_3_[76325]: shutting down. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: ******************************* 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_4_[76326]: starting. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_4_[76326]: rados_pool_create. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_4_[76326]: rados_ioctx_create. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: process_4_[76326]: shutting down. 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: ******************************* 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: ******************************* 2024-04-14T15:12:49.279 INFO:tasks.workunit.client.0.smithi045.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: osdmap e77: 8 total, 8 up, 8 in 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: osdmap e77: 8 total, 8 up, 8 in 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: pgmap v73: 964 pgs: 832 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.3 deep-scrub starts 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.3 deep-scrub ok 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.6 deep-scrub starts 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.6 deep-scrub ok 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.8 deep-scrub starts 2024-04-14T15:12:49.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.8 deep-scrub ok 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.9 deep-scrub starts 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: 15.9 deep-scrub ok 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/769508512' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: pgmap v73: 964 pgs: 832 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.3 deep-scrub starts 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.3 deep-scrub ok 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.6 deep-scrub starts 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.6 deep-scrub ok 2024-04-14T15:12:49.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.8 deep-scrub starts 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.8 deep-scrub ok 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.9 deep-scrub starts 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: 15.9 deep-scrub ok 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/769508512' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:49.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: osdmap e78: 8 total, 8 up, 8 in 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.756 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: osdmap e78: 8 total, 8 up, 8 in 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.757 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:49.758 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:49 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: osdmap e77: 8 total, 8 up, 8 in 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: pgmap v73: 964 pgs: 832 unknown, 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.3 deep-scrub starts 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.3 deep-scrub ok 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.6 deep-scrub starts 2024-04-14T15:12:49.862 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.6 deep-scrub ok 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.8 deep-scrub starts 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.8 deep-scrub ok 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.9 deep-scrub starts 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: 15.9 deep-scrub ok 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/769508512' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:49.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: osdmap e78: 8 total, 8 up, 8 in 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:49.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:49.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:49.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:49.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:49 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.288 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: Running main() from gmock_main.cc 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [----------] Global test environment set-up. 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (1531 ms) 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (1954 ms) 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3042 ms) 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [----------] 3 tests from NeoRadosCmd (6527 ms total) 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [----------] Global test environment tear-down 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [==========] 3 tests from 1 test suite ran. (6527 ms total) 2024-04-14T15:12:50.289 INFO:tasks.workunit.client.0.smithi045.stdout: cmd: [ PASSED ] 3 tests. 2024-04-14T15:12:50.329 INFO:tasks.workunit.client.0.smithi045.stdout:12:46.477163+0000 mon.a [INF] from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:50.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.477534+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.477871+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.480086+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.480402+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.480492+0000 mon.a [INF] from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi045-74884-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:46.481103+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.236453+0000 mon.a [INF] from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.236508+0000 mon.a [INF] from='client.25132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.236555+0000 mon.a [INF] from='client.25090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.236603+0000 mon.a [INF] from='client.35114 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi045-74470-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.271170+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.276256+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.278277+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]: dispatch 2024-04-14T15:12:50.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.289440+0000 mon.a [INF] from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.289659+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.290041+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.314095+0000 mon.a [INF] from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.343836+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.346090+0000 mon.a [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.352059+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.352742+0000 mon.a [INF] from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.356758+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.357232+0000 mon.a [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.358636+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.359323+0000 mon.a [INF] from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.361337+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.361851+0000 mon.a [INF] from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.371528+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:48.372207+0000 mon.a [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240321+0000 mon.a [INF] from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi045-74511-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240395+0000 mon.a [INF] from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi045-74524-23", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240492+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi045-74940-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240560+0000 mon.a [INF] from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi045-74884-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240624+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/575524427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi045-75731-1"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240679+0000 mon.a [INF] from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240738+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240801+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240868+0000 mon.a [INF] from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.240957+0000 mon.a [INF] from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi045-74591-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.246485+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/324731881' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:50.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.246600+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.248085+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.248416+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/1856105077' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:50.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.249818+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.274858+0000 mon.a [INF] from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.275076+0000 mon.a [INF] from='client.34901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.275588+0000 mon.a [INF] from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.275951+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.276143+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.276404+0000 mon.a [INF] from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.324491+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:49.348257+0000 mon.a [INF] from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.246265+0000 mon.a [INF] from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.246354+0000 mon.a [INF] from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.246439+0000 mon.a [INF] from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:50.384 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.246528+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:50.385 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.246617+0000 mon.a [INF] from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.385 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.255251+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.385 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.255578+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.276976+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/2989456938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","p api_io_pp: Running main() from gmock_main.cc 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [==========] Running 38 tests from 2 test suites. 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] Global test environment set-up. 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] 21 tests from LibRadosIoPP 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: seed 74524 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (384 ms) 2024-04-14T15:12:50.403 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (8 ms) 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadExtentArrayOpPP 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadExtentArrayOpPP (365 ms) 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-04-14T15:12:50.404 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (1 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (3 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-04-14T15:12:50.405 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (3 ms) 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-04-14T15:12:50.406 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (10 ms) 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (1 ms) 2024-04-14T15:12:50.407 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] 21 tests from LibRadosIoPP (816 ms total) 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1121 ms) 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (25 ms) 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-04-14T15:12:50.408 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-04-14T15:12:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.7 deep-scrub starts 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.7 deep-scrub ok 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.0 deep-scrub starts 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.0 deep-scrub ok 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.5 deep-scrub starts 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: 15.5 deep-scrub ok 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: osdmap e79: 8 total, 8 up, 8 in 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2989456938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25351 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.7 deep-scrub starts 2024-04-14T15:12:50.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.7 deep-scrub ok 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.0 deep-scrub starts 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.0 deep-scrub ok 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.5 deep-scrub starts 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: 15.5 deep-scrub ok 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: osdmap e79: 8 total, 8 up, 8 in 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2989456938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25351 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1808487735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.35345 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: threexx 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1808487735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.35345 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: threexx 2024-04-14T15:12:50.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:50 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.7 deep-scrub starts 2024-04-14T15:12:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.7 deep-scrub ok 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.0 deep-scrub starts 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.0 deep-scrub ok 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.5 deep-scrub starts 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: 15.5 deep-scrub ok 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi045-74490-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.34901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi045-75769-1"}]': finished 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.34763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi045-75020-1"}]': finished 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:50.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi045-75731-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3487504291' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: osdmap e79: 8 total, 8 up, 8 in 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2989456938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25351 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1808487735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.35345 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:50.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:50.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: threexx 2024-04-14T15:12:50.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:50 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:51.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.4 deep-scrub starts 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.4 deep-scrub ok 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: pgmap v76: 868 pgs: 32 creating+peering, 2 active, 36 creating+activating, 446 active+clean, 352 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: fourxx 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.2 deep-scrub starts 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.2 deep-scrub ok 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.1 deep-scrub starts 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: 15.1 deep-scrub ok 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/540110389' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25351 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: osdmap e80: 8 total, 8 up, 8 in 2024-04-14T15:12:51.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.4 deep-scrub starts 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.4 deep-scrub ok 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: pgmap v76: 868 pgs: 32 creating+peering, 2 active, 36 creating+activating, 446 active+clean, 352 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: fourxx 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.2 deep-scrub starts 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.2 deep-scrub ok 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.1 deep-scrub starts 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: 15.1 deep-scrub ok 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/540110389' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:51.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25351 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: osdmap e80: 8 total, 8 up, 8 in 2024-04-14T15:12:51.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:51.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:51 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.4 deep-scrub starts 2024-04-14T15:12:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.4 deep-scrub ok 2024-04-14T15:12:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: pgmap v76: 868 pgs: 32 creating+peering, 2 active, 36 creating+activating, 446 active+clean, 352 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-04-14T15:12:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: fourxx 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.2 deep-scrub starts 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.2 deep-scrub ok 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.1 deep-scrub starts 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: 15.1 deep-scrub ok 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/540110389' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi045-74591-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi045-74587-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:51.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25084 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]': finished 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25351 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35345 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: osdmap e80: 8 total, 8 up, 8 in 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:51.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:51.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:51.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:51.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:51 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.276 INFO:tasks.workunit.client.0.smithi045.stdout: list: Running main() from gmock_main.cc 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [----------] Global test environment set-up. 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [----------] 3 tests from NeoradosList 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ OK ] NeoradosList.ListObjects (2431 ms) 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ OK ] NeoradosList.ListObjectsNS (3011 ms) 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ OK ] NeoradosList.ListObjectsMany (3024 ms) 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [----------] 3 tests from NeoradosList (8466 ms total) 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [----------] Global test environment tear-down 2024-04-14T15:12:52.277 INFO:tasks.workunit.client.0.smithi045.stdout: list: [==========] 3 tests from 1 test suite ran. (8466 ms total) 2024-04-14T15:12:52.278 INFO:tasks.workunit.client.0.smithi045.stdout: list: [ PASSED ] 3 tests. 2024-04-14T15:12:52.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: osdmap e81: 8 total, 8 up, 8 in 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: osdmap e81: 8 total, 8 up, 8 in 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:52 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25117 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-04-14T15:12:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-4-cache", "mode": "writeback"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2069627014' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4262094403' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4283903695' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3072160133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: osdmap e81: 8 total, 8 up, 8 in 2024-04-14T15:12:52.759 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]: dispatch 2024-04-14T15:12:52.760 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]: dispatch 2024-04-14T15:12:52.760 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]: dispatch 2024-04-14T15:12:52.760 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]: dispatch 2024-04-14T15:12:52.760 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.760 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:52 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:52.868 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [----------] Global test environment set-up. 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [----------] 4 tests from LibRadosService 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5041 ms) 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ OK ] LibRadosService.RegisterLate (15 ms) 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: cluster: 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: id: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: health: HEALTH_WARN 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 11 pool(s) do not have an application enabled 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: services: 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-04-14T15:12:52.869 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: data: 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pools: 25 pools, 668 pgs 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: objects: 198 objects, 582 KiB 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: usage: 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pgs: 80.240% pgs unknown 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 536 unknown 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 132 active+clean 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: io: 2024-04-14T15:12:52.870 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: cluster: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: id: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: health: HEALTH_WARN 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 15 pool(s) do not have an application enabled 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: services: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: data: 2024-04-14T15:12:52.871 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pools: 32 pools, 964 pgs 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: objects: 198 objects, 582 KiB 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: usage: 230 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pgs: 86.307% pgs unknown 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 832 unknown 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 132 active+clean 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: cluster: 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: id: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: health: HEALTH_WARN 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 1 cache pools are missing hit_sets 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 16 pool(s) do not have an application enabled 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: services: 2024-04-14T15:12:52.872 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: data: 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pools: 32 pools, 868 pgs 2024-04-14T15:12:52.873 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: objects: 255 objects, 145 MiB 2024-04-14T15:12:53.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: Running main() from gmock_main.cc 2024-04-14T15:12:53.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-04-14T15:12:53.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] Global test environment set-up. 2024-04-14T15:12:53.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-04-14T15:12:53.283 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStat.Stat (390 ms) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (10 ms) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStat.StatNS (17 ms) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] 5 tests from LibRadosStat (421 ms total) 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-04-14T15:12:53.284 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-04-14T15:12:53.285 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1133 ms) 2024-04-14T15:12:53.285 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (10 ms) 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1145 ms total) 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [----------] Global test environment tear-down 2024-04-14T15:12:53.345 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (9733 ms total) 2024-04-14T15:12:53.346 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat: [ PASSED ] 9 tests. 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Running main() from gmock_main.cc 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [----------] Global test environment set-up. 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: handle_notify cookie 94703317197648 notify_id 317827579904 notifier_gid 24928 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (2378 ms) 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Trying... 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: handle_notify cookie 94703329898736 notify_id 330712481794 notifier_gid 35090 2024-04-14T15:12:53.347 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Waiting for 3.000000000s 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Timed out. 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Flushing... 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: Flushed... 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7035 ms) 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (9413 ms total) 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [----------] Global test environment tear-down 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (9413 ms total) 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: watch_notify: [ PASSED ] 2 tests. 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout:api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-04-14T15:12:53.348 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (5 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-04-14T15:12:53.349 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (10 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (6 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (5 ms) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1221 ms total) 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: 2024-04-14T15:12:53.350 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [----------] Global test environment tear-down 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [==========] 38 tests from 2 test suites ran. (9852 ms total) 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io_pp: [ PASSED ] 38 tests. 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: Running main() from gmock_main.cc 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] Global test environment set-up. 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (395 ms) 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: no timeout :/ 2024-04-14T15:12:53.351 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: no timeout :/ 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: no timeout :/ 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: no timeout :/ 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: no timeout :/ 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (55 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (8 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.Checksum (6 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (6 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-04-14T15:12:53.352 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.RmXattr (13 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] 14 tests from LibRadosIo (516 ms total) 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-04-14T15:12:53.353 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1112 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (19 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (10 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (8 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (6 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-04-14T15:12:53.354 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (8 ms) 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1176 ms total) 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [----------] Global test environment tear-down 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [==========] 24 tests from 2 test suites ran. (9868 ms total) 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_io: [ PASSED ] 24 tests. 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-04-14T15:12:53.355 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: seed 74940 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (379 ms) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (16 ms) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (7 ms) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (21 ms) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (423 ms total) 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: 2024-04-14T15:12:53.356 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1133 ms) 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (10 ms) 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (2 ms) 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (6 ms) 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1151 ms total) 2024-04-14T15:12:53.357 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: 2024-04-14T15:12:53.358 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-04-14T15:12:53.358 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (9720 ms total) 2024-04-14T15:12:53.358 INFO:tasks.workunit.client.0.smithi045.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-04-14T15:12:53.393 INFO:tasks.workunit.client.0.smithi045.stdout:ool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.393 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.284733+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-4", "overlaypool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.285093+0000 mon.a [INF] from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi045-75731-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.290119+0000 mon.a [INF] from='client.25084 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi045-74490-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.290583+0000 mon.a [INF] from='client.25351 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.291985+0000 mon.a [INF] from='client.? v1:172.21.15.45:0/2614135761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi045-74470-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.298376+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.300640+0000 mon.a [INF] from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi045-75769-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.304478+0000 mon.c [INF] from='client.? v1:172.21.15.45:0/1808487735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.305088+0000 mon.a [INF] from='client.35345 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.328614+0000 mon.b [INF] from='client.? v1:172.21.15.45:0/4109599910' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.328788+0000 client.admin [INF] threexx 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: got: 2024-04-14T15:12:50.329226+0000 mon.a [INF] from='client.25117 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (7137 ms) 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (9808 ms total) 2024-04-14T15:12:53.394 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: 2024-04-14T15:12:53.395 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [----------] Global test environment tear-down 2024-04-14T15:12:53.395 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (9809 ms total) 2024-04-14T15:12:53.395 INFO:tasks.workunit.client.0.smithi045.stdout: api_cmd: [ PASSED ] 4 tests. 2024-04-14T15:12:53.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: pgmap v79: 700 pgs: 32 creating+peering, 2 active, 25 creating+activating, 425 active+clean, 216 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]: dispatch 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1857294165' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]': finished 2024-04-14T15:12:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]': finished 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: osdmap e82: 8 total, 8 up, 8 in 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2489021179' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: pgmap v79: 700 pgs: 32 creating+peering, 2 active, 25 creating+activating, 425 active+clean, 216 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]: dispatch 2024-04-14T15:12:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1857294165' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]': finished 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: osdmap e82: 8 total, 8 up, 8 in 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2489021179' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3130956713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3492186304' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35417 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[29624]: from='client.35408 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3130956713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3492186304' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35417 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:53 smithi045 ceph-mon[37795]: from='client.35408 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:12:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: pgmap v79: 700 pgs: 32 creating+peering, 2 active, 25 creating+activating, 425 active+clean, 216 unknown; 145 MiB data, 406 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-04-14T15:12:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]: dispatch 2024-04-14T15:12:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]: dispatch 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1857294165' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi045-75769-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi045-74511-16"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.25099 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi045-74524-23"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi045-74940-7"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.25096 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi045-74884-7"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-75020-6", "pg_num": 4}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-4"}]': finished 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: osdmap e82: 8 total, 8 up, 8 in 2024-04-14T15:12:53.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2489021179' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3130956713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3492186304' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35417 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35405 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:53.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:53 smithi071 ceph-mon[33150]: from='client.35408 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35417 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35408 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: osdmap e83: 8 total, 8 up, 8 in 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:12:54.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35417 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35408 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: osdmap e83: 8 total, 8 up, 8 in 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2"}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:54 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1951959466' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-4", "tierpool": "test-rados-api-smithi045-74757-4-cache"}]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35417 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35405 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi045-74470-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35408 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: osdmap e83: 8 total, 8 up, 8 in 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2"}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:12:54.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:54 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:55.970 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: Running main() from gmock_main.cc 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [----------] Global test environment set-up. 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (5437 ms) 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (6728 ms) 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (12165 ms total) 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [----------] Global test environment tear-down 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (12165 ms total) 2024-04-14T15:12:55.971 INFO:tasks.workunit.client.0.smithi045.stdout: ec_io: [ PASSED ] 2 tests. 2024-04-14T15:12:56.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: pgmap v82: 832 pgs: 204 creating+peering, 532 active+clean, 96 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: osdmap e84: 8 total, 8 up, 8 in 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: pgmap v82: 832 pgs: 204 creating+peering, 532 active+clean, 96 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:56.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: osdmap e84: 8 total, 8 up, 8 in 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.250 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: pgmap v82: 832 pgs: 204 creating+peering, 532 active+clean, 96 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "overlaypool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi045-74657-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: osdmap e84: 8 total, 8 up, 8 in 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/328703526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:56.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: Running main() from gmock_main.cc 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [----------] Global test environment set-up. 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolList (1425 ms) 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (1985 ms) 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-04-14T15:12:56.982 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2007 ms) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4022 ms) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1690 ms) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2009 ms) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [----------] 6 tests from NeoRadosPools (13138 ms total) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [----------] Global test environment tear-down 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [==========] 6 tests from 1 test suite ran. (13138 ms total) 2024-04-14T15:12:56.983 INFO:tasks.workunit.client.0.smithi045.stdout: pool: [ PASSED ] 6 tests. 2024-04-14T15:12:56.984 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: Running main() from gmock_main.cc 2024-04-14T15:12:56.984 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-04-14T15:12:56.984 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] Global test environment set-up. 2024-04-14T15:12:56.984 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (375 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.LockShared (18 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1010 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1004 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.Unlock (14 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-04-14T15:12:56.985 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (2 ms) 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] 8 tests from LibRadosLock (2432 ms total) 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1161 ms) 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5 ms) 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1037 ms) 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-04-14T15:12:56.986 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1033 ms) 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-04-14T15:12:56.987 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3250 ms total) 2024-04-14T15:12:56.988 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: 2024-04-14T15:12:56.988 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [----------] Global test environment tear-down 2024-04-14T15:12:56.988 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (13514 ms total) 2024-04-14T15:12:56.988 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock: [ PASSED ] 16 tests. 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: seed 74587 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (412 ms) 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (8 ms) 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1009 ms) 2024-04-14T15:12:56.989 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1034 ms) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (12 ms) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (6 ms) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (6 ms) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (2 ms) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2490 ms total) 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-04-14T15:12:56.990 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1159 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1050 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1013 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-04-14T15:12:56.991 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3243 ms total) 2024-04-14T15:12:56.992 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: 2024-04-14T15:12:56.992 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-04-14T15:12:56.992 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (13531 ms total) 2024-04-14T15:12:56.992 INFO:tasks.workunit.client.0.smithi045.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-04-14T15:12:56.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:12:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:12:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: osdmap e85: 8 total, 8 up, 8 in 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2362624105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2419342597' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1522695591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: osdmap e86: 8 total, 8 up, 8 in 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.361 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:57 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: osdmap e85: 8 total, 8 up, 8 in 2024-04-14T15:12:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2362624105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2419342597' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1522695591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:12:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.25342 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi045-75731-2"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-75020-6", "mode": "writeback"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3221447850' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi045-74775-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3379015295' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3418967576' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3004209135' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: osdmap e85: 8 total, 8 up, 8 in 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2362624105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2419342597' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1522695591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.25483 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:57.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: osdmap e86: 8 total, 8 up, 8 in 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.25486 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:57.502 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi045-74591-10"}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35351 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi045-75769-2"}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.25483 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35186 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi045-74587-10"}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.25486 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi045-74470-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: osdmap e86: 8 total, 8 up, 8 in 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:57.503 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.504 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:57.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4125340382' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:57.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:12:57.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:57 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: pgmap v85: 712 pgs: 4 creating+peering, 420 active+clean, 288 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: osdmap e87: 8 total, 8 up, 8 in 2024-04-14T15:12:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:12:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: pgmap v85: 712 pgs: 4 creating+peering, 420 active+clean, 288 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: osdmap e87: 8 total, 8 up, 8 in 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:12:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: pgmap v85: 712 pgs: 4 creating+peering, 420 active+clean, 288 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.35375 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi045-74657-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi045-75769-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: osdmap e87: 8 total, 8 up, 8 in 2024-04-14T15:12:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:58.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:12:58.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.105 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1134 ms) 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1138 ms total) 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: 2024-04-14T15:12:59.106 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (155 ms) 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3640 ms) 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (10 ms) 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220162 notifier_gid 35360 2024-04-14T15:12:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220163 notifier_gid 35360 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (4 ms) 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220163 notifier_gid 35360 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220163 notifier_gid 35360 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220164 notifier_gid 35360 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-04-14T15:12:59.108 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584289632 notify_id 365072220164 notifier_gid 35360 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: trying... 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584316112 notify_id 365072220161 notifier_gid 35360 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: timed out 2024-04-14T15:12:59.109 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: flushing 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: osdmap e88: 8 total, 8 up, 8 in 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1829310381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2831335420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.25531 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.25537 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2788329578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:12:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4086925878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: osdmap e88: 8 total, 8 up, 8 in 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1829310381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2831335420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.25531 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.25537 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2788329578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4086925878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi045-74975-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: osdmap e88: 8 total, 8 up, 8 in 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1829310381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2831335420' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.25531 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.25537 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2788329578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:12:59.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:12:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4086925878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: pgmap v88: 488 pgs: 4 creating+peering, 356 active+clean, 128 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.25552 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: pool 'PoolQuotaPP_smithi045-74490-3' is full (reached quota's max_bytes: 4 KiB) 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.25537 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.25552 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: osdmap e89: 8 total, 8 up, 8 in 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: pgmap v88: 488 pgs: 4 creating+peering, 356 active+clean, 128 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.25552 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: pool 'PoolQuotaPP_smithi045-74490-3' is full (reached quota's max_bytes: 4 KiB) 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.25537 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.25552 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: osdmap e89: 8 total, 8 up, 8 in 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: pgmap v88: 488 pgs: 4 creating+peering, 356 active+clean, 128 unknown; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.35546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.25552 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: pool 'PoolQuotaPP_smithi045-74490-3' is full (reached quota's max_bytes: 4 KiB) 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:13:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi045-75769-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.25531 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi045-74876-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.25537 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.35546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.25552 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi045-74470-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: osdmap e89: 8 total, 8 up, 8 in 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:00.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[29624]: osdmap e90: 8 total, 8 up, 8 in 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:01.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:01.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[37795]: osdmap e90: 8 total, 8 up, 8 in 2024-04-14T15:13:01.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:01.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:01 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:01 smithi071 ceph-mon[33150]: osdmap e90: 8 total, 8 up, 8 in 2024-04-14T15:13:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:01 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: pgmap v91: 760 pgs: 64 creating+peering, 392 active+clean, 304 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 4 op/s 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: osdmap e91: 8 total, 8 up, 8 in 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/706607467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.28543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: pgmap v91: 760 pgs: 64 creating+peering, 392 active+clean, 304 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 4 op/s 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:02.403 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: osdmap e91: 8 total, 8 up, 8 in 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/706607467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.28543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.404 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:02 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: pgmap v91: 760 pgs: 64 creating+peering, 392 active+clean, 304 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 4 op/s 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: osdmap e91: 8 total, 8 up, 8 in 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/706607467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.28543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2901998255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:02 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]: dispatch 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_wat api_watch_notify: Running main() from gmock_main.cc 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify_test_cb 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (391 ms) 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94097949137072 err -107 2024-04-14T15:13:03.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (11 ms) 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94097949139312 err -107 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1012 ms) 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547200 cookie 94097949143648 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (5 ms) 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547201 cookie 94097949143648 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (5 ms) 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-04-14T15:13:03.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547200 cookie 94097949878080 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (5 ms) 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547201 cookie 94097949855440 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547201 cookie 94097949878080 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (6 ms) 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 322122547202 cookie 94097949878080 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 326417514499 cookie 94097949878080 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94097949855440 err -107 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_cb from 24763 notify_id 356482285573 cookie 94097949855440 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-04-14T15:13:03.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94097949857568 err -107 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10451 ms total) 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: watch_notify_test_cb 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1168 ms) 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1168 ms total) 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-04-14T15:13:03.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (19448 ms total) 2024-04-14T15:13:03.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-04-14T15:13:03.495 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: from='client.28543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-mon[33150]: osdmap e92: 8 total, 8 up, 8 in 2024-04-14T15:13:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: from='client.28543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[37795]: osdmap e92: 8 total, 8 up, 8 in 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1898159165' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi045-74470-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: from='client.28543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-75200-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: from='client.35510 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi045-74975-12"}]': finished 2024-04-14T15:13:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:03 smithi045 ceph-mon[29624]: osdmap e92: 8 total, 8 up, 8 in 2024-04-14T15:13:03.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: pgmap v94: 688 pgs: 32 creating+peering, 392 active+clean, 264 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 8.0 KiB/s wr, 7 op/s 2024-04-14T15:13:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: osdmap e93: 8 total, 8 up, 8 in 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: pgmap v94: 688 pgs: 32 creating+peering, 392 active+clean, 264 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 8.0 KiB/s wr, 7 op/s 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:04.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: osdmap e93: 8 total, 8 up, 8 in 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:04.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: pgmap v94: 688 pgs: 32 creating+peering, 392 active+clean, 264 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 8.0 KiB/s wr, 7 op/s 2024-04-14T15:13:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: osdmap e93: 8 total, 8 up, 8 in 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:04.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]: dispatch 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:05.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: osdmap e94: 8 total, 8 up, 8 in 2024-04-14T15:13:05.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:05.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: osdmap e94: 8 total, 8 up, 8 in 2024-04-14T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4079818414' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-7"}]': finished 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: osdmap e94: 8 total, 8 up, 8 in 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]: dispatch 2024-04-14T15:13:05.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:06.060 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: Running main() from gmock_main.cc 2024-04-14T15:13:06.060 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [----------] Global test environment set-up. 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (6438 ms) 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (6719 ms) 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9049 ms) 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [----------] 3 tests from NeoradosECList (22206 ms total) 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [----------] Global test environment tear-down 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (22206 ms total) 2024-04-14T15:13:06.061 INFO:tasks.workunit.client.0.smithi045.stdout: ec_list: [ PASSED ] 3 tests. 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: pgmap v97: 648 pgs: 64 creating+peering, 552 active+clean, 32 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:06 smithi071 ceph-mon[33150]: osdmap e95: 8 total, 8 up, 8 in 2024-04-14T15:13:06.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: pgmap v97: 648 pgs: 64 creating+peering, 552 active+clean, 32 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-14T15:13:06.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:06.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:13:06.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:06.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[29624]: osdmap e95: 8 total, 8 up, 8 in 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: pgmap v97: 648 pgs: 64 creating+peering, 552 active+clean, 32 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3804073819' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi045-75769-3"}]': finished 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1625415542' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi045-74470-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:06.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-mon[37795]: osdmap e95: 8 total, 8 up, 8 in 2024-04-14T15:13:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[29624]: osdmap e96: 8 total, 8 up, 8 in 2024-04-14T15:13:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:07.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:07.425 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:07.425 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[37795]: osdmap e96: 8 total, 8 up, 8 in 2024-04-14T15:13:07.425 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:07.425 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:07.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:07.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:07 smithi071 ceph-mon[33150]: osdmap e96: 8 total, 8 up, 8 in 2024-04-14T15:13:07.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:07.447 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: Running main() from gmock_main.cc 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [----------] Global test environment set-up. 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (2382 ms) 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (3019 ms) 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (3012 ms) 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-04-14T15:13:08.061 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (2695 ms) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3028 ms) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (2994 ms) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (7059 ms) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (24189 ms total) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [----------] Global test environment tear-down 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (24189 ms total) 2024-04-14T15:13:08.062 INFO:tasks.workunit.client.0.smithi045.stdout: write_operations: [ PASSED ] 7 tests. 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-04-14T15:13:08.080 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (2611 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3006 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3019 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2693 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3025 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3002 ms) 2024-04-14T15:13:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (7079 ms) 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (24435 ms total) 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (24435 ms total) 2024-04-14T15:13:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-04-14T15:13:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: pgmap v100: 712 pgs: 32 creating+peering, 488 active+clean, 192 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-14T15:13:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: osdmap e97: 8 total, 8 up, 8 in 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2611782643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.34546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: pgmap v100: 712 pgs: 32 creating+peering, 488 active+clean, 192 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-14T15:13:08.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: osdmap e97: 8 total, 8 up, 8 in 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2611782643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.34546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:08 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: pgmap v100: 712 pgs: 32 creating+peering, 488 active+clean, 192 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 3.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: osdmap e97: 8 total, 8 up, 8 in 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2611782643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.34546 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:08 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]: dispatch 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.34546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]': finished 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: osdmap e98: 8 total, 8 up, 8 in 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/668497309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.44753 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.34546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]': finished 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: osdmap e98: 8 total, 8 up, 8 in 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/668497309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.44753 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.34546 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3074284640' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi045-74470-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4"}]': finished 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: osdmap e98: 8 total, 8 up, 8 in 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/668497309' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.44753 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:09.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:09.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]: dispatch 2024-04-14T15:13:09.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:10.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: pgmap v103: 584 pgs: 32 creating+peering, 392 active+clean, 160 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:10.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:10.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.44753 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: osdmap e99: 8 total, 8 up, 8 in 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:10.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: pgmap v103: 584 pgs: 32 creating+peering, 392 active+clean, 160 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:10.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.44753 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: osdmap e99: 8 total, 8 up, 8 in 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: pgmap v103: 584 pgs: 32 creating+peering, 392 active+clean, 160 unknown; 145 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.44753 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi045-75020-4", "tierpool": "test-rados-api-smithi045-75020-6"}]': finished 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: osdmap e99: 8 total, 8 up, 8 in 2024-04-14T15:13:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2864590449' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: pgmap v106: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]': finished 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: osdmap e100: 8 total, 8 up, 8 in 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1884261832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3960808041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.507 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.37639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.508 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.46724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.508 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.508 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.508 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:11.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: pgmap v106: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-14T15:13:11.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:11.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]': finished 2024-04-14T15:13:11.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: osdmap e100: 8 total, 8 up, 8 in 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1884261832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3960808041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.37639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.46724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: pgmap v106: 488 pgs: 96 unknown, 392 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-9", "mode": "writeback"}]': finished 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.35360 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi045-75020-6", "pool2": "test-rados-api-smithi045-75020-6", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: osdmap e100: 8 total, 8 up, 8 in 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1884261832' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3960808041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.37639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.46724 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout:ch_notify_pp: flushed 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3005 ms) 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: trying... 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584316112 notify_id 377957122053 notifier_gid 35360 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: timed out 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: flushing 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: flushed 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3016 ms) 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-04-14T15:13:12.031 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: List watches 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify2 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584334992 notify_id 390842023942 notifier_gid 35360 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify2 done 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: watch_check 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: unwatch2 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: flushing 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: done 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3026 ms) 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: List watches 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify2 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: handle_notify cookie 94604584334992 notify_id 403726925826 notifier_gid 35360 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: notify2 done 2024-04-14T15:13:12.032 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: watch_check 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: unwatch2 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: flushing 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: done 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3015 ms) 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (15899 ms total) 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (28430 ms total) 2024-04-14T15:13:12.033 INFO:tasks.workunit.client.0.smithi045.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.37639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.46724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: osdmap e101: 8 total, 8 up, 8 in 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:12.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:12 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: usage: 406 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: pgs: 40.553% pgs unknown 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 7.834% pgs not active 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 446 active+clean 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 352 unknown 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 36 creating+activating 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 32 creating+peering 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2 active 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: io: 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: client: 12 MiB/s rd, 37 MiB/s wr, 256 op/s rd, 291 op/s wr 2024-04-14T15:13:12.942 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4185 ms) 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ RUN ] LibRadosService.Status 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ OK ] LibRadosService.Status (20024 ms) 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [----------] 4 tests from LibRadosService (29265 ms total) 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [----------] Global test environment tear-down 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29265 ms total) 2024-04-14T15:13:12.943 INFO:tasks.workunit.client.0.smithi045.stdout: api_service: [ PASSED ] 4 tests. 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.37639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.46724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: osdmap e101: 8 total, 8 up, 8 in 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:13.004 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.37639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi045-74470-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.46724 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi045-74490-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: osdmap e101: 8 total, 8 up, 8 in 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]: dispatch 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:13.005 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:12 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:13.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: pgmap v109: 580 pgs: 224 unknown, 356 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: osdmap e102: 8 total, 8 up, 8 in 2024-04-14T15:13:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: pgmap v109: 580 pgs: 224 unknown, 356 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: osdmap e102: 8 total, 8 up, 8 in 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: pgmap v109: 580 pgs: 224 unknown, 356 active+clean; 145 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi045-74775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-9"}]': finished 2024-04-14T15:13:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: osdmap e102: 8 total, 8 up, 8 in 2024-04-14T15:13:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:14.056 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-04-14T15:13:14.056 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [----------] Global test environment set-up. 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5052 ms) 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (45 ms) 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20030 ms) 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: attempt 0 of 20 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5223 ms) 2024-04-14T15:13:14.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30350 ms total) 2024-04-14T15:13:14.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: 2024-04-14T15:13:14.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [----------] Global test environment tear-down 2024-04-14T15:13:14.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30350 ms total) 2024-04-14T15:13:14.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-04-14T15:13:14.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:14.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2272115095' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: osdmap e103: 8 total, 8 up, 8 in 2024-04-14T15:13:14.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3052001927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.40750 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/803599149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.49622 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:14.859 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2272115095' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: osdmap e103: 8 total, 8 up, 8 in 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3052001927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.40750 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/803599149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.49622 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:14.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2272115095' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='mon.0 v1:172.21.15.45:0/2218894212' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: osdmap e103: 8 total, 8 up, 8 in 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3052001927' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.40750 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/803599149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.49622 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:13:14.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:15.000 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: pgmap v112: 524 pgs: 8 creating+peering, 192 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.40750 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.49622 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: osdmap e104: 8 total, 8 up, 8 in 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: pgmap v112: 524 pgs: 8 creating+peering, 192 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.40750 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.49622 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: osdmap e104: 8 total, 8 up, 8 in 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: pgmap v112: 524 pgs: 8 creating+peering, 192 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.40750 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi045-74490-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74771-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.49622 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi045-74470-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: osdmap e104: 8 total, 8 up, 8 in 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:15.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: Running main() from gmock_main.cc 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [----------] Global test environment set-up. 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.Version (1433 ms) 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-04-14T15:13:16.004 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (1982 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.LongName (3040 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (3006 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (2710 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3006 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.Exec (2997 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3011 ms) 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-04-14T15:13:16.005 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3028 ms) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2976 ms) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2022 ms) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (2957 ms) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [----------] 12 tests from NeoRadosMisc (32168 ms total) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [----------] Global test environment tear-down 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [==========] 12 tests from 1 test suite ran. (32168 ms total) 2024-04-14T15:13:16.006 INFO:tasks.workunit.client.0.smithi045.stdout: misc: [ PASSED ] 12 tests. 2024-04-14T15:13:16.979 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:16.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:16.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[29624]: osdmap e105: 8 total, 8 up, 8 in 2024-04-14T15:13:16.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:16.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:16.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:17.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:17.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[37795]: osdmap e105: 8 total, 8 up, 8 in 2024-04-14T15:13:17.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:17.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:17.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:16 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:16 smithi071 ceph-mon[33150]: osdmap e105: 8 total, 8 up, 8 in 2024-04-14T15:13:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:16 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]: dispatch 2024-04-14T15:13:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: pgmap v115: 460 pgs: 8 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]': finished 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: osdmap e106: 8 total, 8 up, 8 in 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2089200705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2308479411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.43945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.43948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: pgmap v115: 460 pgs: 8 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-14T15:13:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]': finished 2024-04-14T15:13:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: osdmap e106: 8 total, 8 up, 8 in 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2089200705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2308479411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.43945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.43948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: pgmap v115: 460 pgs: 8 creating+peering, 128 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache", "force_nonempty":""}]': finished 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: osdmap e106: 8 total, 8 up, 8 in 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2089200705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2308479411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.43945 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.43948 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:18.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.43945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.43948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: osdmap e107: 8 total, 8 up, 8 in 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:19.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.43945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.43948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: osdmap e107: 8 total, 8 up, 8 in 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.43945 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi045-74470-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74771-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.43948 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi045-74490-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: osdmap e107: 8 total, 8 up, 8 in 2024-04-14T15:13:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3743423482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:19.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:19.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:20.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-04-14T15:13:20.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] Global test environment set-up. 2024-04-14T15:13:20.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-04-14T15:13:20.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-04-14T15:13:20.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:12:43.552+0000 7f4a61444780 -1 failed for service _ceph-mon._tcp 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:12:43.552+0000 7f4a61444780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (60 ms) 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2018 ms) 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2078 ms total) 2024-04-14T15:13:20.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: started 0x556247a09630 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: started 0x556247d50de0 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: started 2 aios 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: waiting 0x556247a09630 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: waiting 0x556247d50de0 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: done. 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (5712 ms) 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (5713 ms total) 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-04-14T15:13:20.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.Exec (107 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (8 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4584 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (21273 ms) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] 8 tests from LibRadosMisc (25975 ms total) 2024-04-14T15:13:20.061 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: 2024-04-14T15:13:20.062 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [----------] Global test environment tear-down 2024-04-14T15:13:20.062 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (36547 ms total) 2024-04-14T15:13:20.062 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc: [ PASSED ] 12 tests. 2024-04-14T15:13:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: pgmap v118: 588 pgs: 8 creating+peering, 256 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]': finished 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: osdmap e108: 8 total, 8 up, 8 in 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.3 deep-scrub starts 2024-04-14T15:13:20.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.3 deep-scrub ok 2024-04-14T15:13:20.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.7 deep-scrub starts 2024-04-14T15:13:20.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.8 deep-scrub starts 2024-04-14T15:13:20.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.8 deep-scrub ok 2024-04-14T15:13:20.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:20 smithi071 ceph-mon[33150]: 15.7 deep-scrub ok 2024-04-14T15:13:20.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: pgmap v118: 588 pgs: 8 creating+peering, 256 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]': finished 2024-04-14T15:13:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: osdmap e108: 8 total, 8 up, 8 in 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.3 deep-scrub starts 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.3 deep-scrub ok 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.7 deep-scrub starts 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.8 deep-scrub starts 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.8 deep-scrub ok 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[29624]: 15.7 deep-scrub ok 2024-04-14T15:13:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: pgmap v118: 588 pgs: 8 creating+peering, 256 unknown, 324 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.0"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.1"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.2"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.3"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.4"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.5"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.6"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.7"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.8"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "15.9"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='client.40717 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74771-10", "tierpool":"test-rados-api-smithi045-74771-10-cache"}]': finished 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: osdmap e108: 8 total, 8 up, 8 in 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]: dispatch 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.3 deep-scrub starts 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.3 deep-scrub ok 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.7 deep-scrub starts 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.8 deep-scrub starts 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.8 deep-scrub ok 2024-04-14T15:13:20.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:20 smithi045 ceph-mon[37795]: 15.7 deep-scrub ok 2024-04-14T15:13:21.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: Running main() from gmock_main.cc 2024-04-14T15:13:21.057 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [----------] Global test environment set-up. 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolList (2727 ms) 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3013 ms) 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3014 ms) 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2701 ms) 2024-04-14T15:13:21.058 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3048 ms) 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4984 ms) 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5056 ms) 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4943 ms) 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8054 ms) 2024-04-14T15:13:21.059 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [----------] 9 tests from LibRadosPools (37541 ms total) 2024-04-14T15:13:21.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: 2024-04-14T15:13:21.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [----------] Global test environment tear-down 2024-04-14T15:13:21.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (37541 ms total) 2024-04-14T15:13:21.060 INFO:tasks.workunit.client.0.smithi045.stdout: api_pool: [ PASSED ] 9 tests. 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.9 deep-scrub starts 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.9 deep-scrub ok 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.6 deep-scrub starts 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.6 deep-scrub ok 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]': finished 2024-04-14T15:13:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: osdmap e109: 8 total, 8 up, 8 in 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1741909348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1784197503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.56090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.56084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.4 deep-scrub starts 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.4 deep-scrub ok 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.0 deep-scrub starts 2024-04-14T15:13:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:21 smithi071 ceph-mon[33150]: 15.0 deep-scrub ok 2024-04-14T15:13:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.9 deep-scrub starts 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.9 deep-scrub ok 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.6 deep-scrub starts 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.6 deep-scrub ok 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]': finished 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: osdmap e109: 8 total, 8 up, 8 in 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1741909348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1784197503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.56090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.56084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.4 deep-scrub starts 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.4 deep-scrub ok 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.0 deep-scrub starts 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[29624]: 15.0 deep-scrub ok 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.9 deep-scrub starts 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.9 deep-scrub ok 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.6 deep-scrub starts 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.6 deep-scrub ok 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-11", "mode": "writeback"}]': finished 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: osdmap e109: 8 total, 8 up, 8 in 2024-04-14T15:13:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1741909348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1784197503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.56090 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.56084 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.4 deep-scrub starts 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.4 deep-scrub ok 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.0 deep-scrub starts 2024-04-14T15:13:21.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:21 smithi045 ceph-mon[37795]: 15.0 deep-scrub ok 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: pgmap v121: 492 pgs: 128 unknown, 364 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.2 MiB/s wr, 3 op/s 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: 15.5 deep-scrub starts 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: 15.5 deep-scrub ok 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: from='client.56090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: from='client.56084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: osdmap e110: 8 total, 8 up, 8 in 2024-04-14T15:13:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: 15.2 deep-scrub starts 2024-04-14T15:13:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:22 smithi071 ceph-mon[33150]: 15.2 deep-scrub ok 2024-04-14T15:13:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: pgmap v121: 492 pgs: 128 unknown, 364 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.2 MiB/s wr, 3 op/s 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: 15.5 deep-scrub starts 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: 15.5 deep-scrub ok 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: from='client.56090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: from='client.56084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: osdmap e110: 8 total, 8 up, 8 in 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: 15.2 deep-scrub starts 2024-04-14T15:13:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[29624]: 15.2 deep-scrub ok 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: pgmap v121: 492 pgs: 128 unknown, 364 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.2 MiB/s wr, 3 op/s 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: 15.5 deep-scrub starts 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: 15.5 deep-scrub ok 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: from='client.56090 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi045-74470-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: from='client.56084 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi045-74490-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: osdmap e110: 8 total, 8 up, 8 in 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:13:22.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:22.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: 15.2 deep-scrub starts 2024-04-14T15:13:22.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:22 smithi045 ceph-mon[37795]: 15.2 deep-scrub ok 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: 15.1 deep-scrub starts 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: 15.1 deep-scrub ok 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: osdmap e111: 8 total, 8 up, 8 in 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: 15.1 deep-scrub starts 2024-04-14T15:13:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: 15.1 deep-scrub ok 2024-04-14T15:13:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: osdmap e111: 8 total, 8 up, 8 in 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: 15.1 deep-scrub starts 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: 15.1 deep-scrub ok 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: osdmap e111: 8 total, 8 up, 8 in 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]: dispatch 2024-04-14T15:13:23.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:23.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: pgmap v124: 364 pgs: 32 unknown, 332 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.3 MiB/s wr, 3 op/s 2024-04-14T15:13:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: osdmap e112: 8 total, 8 up, 8 in 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2270869483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.56258 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: pgmap v124: 364 pgs: 32 unknown, 332 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.3 MiB/s wr, 3 op/s 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: osdmap e112: 8 total, 8 up, 8 in 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2270869483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.56258 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: pgmap v124: 364 pgs: 32 unknown, 332 active+clean; 169 MiB data, 828 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 6.3 MiB/s wr, 3 op/s 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:13:24.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-11"}]': finished 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: osdmap e112: 8 total, 8 up, 8 in 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2270869483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.56258 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:24.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: Running main() from gmock_main.cc 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: [----------] Global test environment set-up. 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: [----------] 14 tests from NeoRadosIo 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-04-14T15:13:24.990 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.Limits (2446 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (3001 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.ReadOp (3026 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.SparseRead (2700 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3032 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (2971 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3023 ms) 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-04-14T15:13:24.991 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3018 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2978 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.Trunc (3007 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.Remove (2994 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3028 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.RmXattr (3021 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2938 ms) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [----------] 14 tests from NeoRadosIo (41183 ms total) 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: 2024-04-14T15:13:24.992 INFO:tasks.workunit.client.0.smithi045.stdout: io: [----------] Global test environment tear-down 2024-04-14T15:13:24.993 INFO:tasks.workunit.client.0.smithi045.stdout: io: [==========] 14 tests from 1 test suite ran. (41184 ms total) 2024-04-14T15:13:24.993 INFO:tasks.workunit.client.0.smithi045.stdout: io: [ PASSED ] 14 tests. 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: Running main() from gmock_main.cc 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [----------] Global test environment set-up. 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (2392 ms) 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (3050 ms) 2024-04-14T15:13:25.014 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (2994 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (2684 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3040 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (2974 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3018 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3014 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2966 ms) 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-04-14T15:13:25.015 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3014 ms) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (2991 ms) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3027 ms) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3037 ms) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (2955 ms) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (41157 ms total) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [----------] Global test environment tear-down 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (41157 ms total) 2024-04-14T15:13:25.016 INFO:tasks.workunit.client.0.smithi045.stdout: read_operations: [ PASSED ] 14 tests. 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.56258 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: osdmap e113: 8 total, 8 up, 8 in 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: osdmap e114: 8 total, 8 up, 8 in 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.56258 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: osdmap e113: 8 total, 8 up, 8 in 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: osdmap e114: 8 total, 8 up, 8 in 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/208175969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.56258 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi045-74470-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: osdmap e113: 8 total, 8 up, 8 in 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3758071171' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]: dispatch 2024-04-14T15:13:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.46610 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi045-74775-10"}]': finished 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: osdmap e114: 8 total, 8 up, 8 in 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:25.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: pgmap v127: 452 pgs: 128 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: osdmap e115: 8 total, 8 up, 8 in 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/746000155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.47695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: pgmap v127: 452 pgs: 128 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: osdmap e115: 8 total, 8 up, 8 in 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/746000155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.47695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: pgmap v127: 452 pgs: 128 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: osdmap e115: 8 total, 8 up, 8 in 2024-04-14T15:13:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/746000155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.47695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:26.998 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:27 smithi071 ceph-mon[33150]: from='client.47695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:27 smithi071 ceph-mon[33150]: osdmap e116: 8 total, 8 up, 8 in 2024-04-14T15:13:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[37795]: from='client.47695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[37795]: osdmap e116: 8 total, 8 up, 8 in 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[29624]: from='client.47695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi045-74490-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3239363221' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi045-74470-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:27 smithi045 ceph-mon[29624]: osdmap e116: 8 total, 8 up, 8 in 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: pgmap v130: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:28 smithi071 ceph-mon[33150]: osdmap e117: 8 total, 8 up, 8 in 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: pgmap v130: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[37795]: osdmap e117: 8 total, 8 up, 8 in 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: pgmap v130: 420 pgs: 96 unknown, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 12 MiB/s wr, 5 op/s 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi045-74775-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:28 smithi045 ceph-mon[29624]: osdmap e117: 8 total, 8 up, 8 in 2024-04-14T15:13:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:29 smithi071 ceph-mon[33150]: osdmap e118: 8 total, 8 up, 8 in 2024-04-14T15:13:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1140052326' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:29 smithi071 ceph-mon[33150]: from='client.47716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[37795]: osdmap e118: 8 total, 8 up, 8 in 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1140052326' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[37795]: from='client.47716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[29624]: osdmap e118: 8 total, 8 up, 8 in 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1140052326' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:29 smithi045 ceph-mon[29624]: from='client.47716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: pgmap v133: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: from='client.47716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:30 smithi071 ceph-mon[33150]: osdmap e119: 8 total, 8 up, 8 in 2024-04-14T15:13:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: pgmap v133: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: from='client.47716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[29624]: osdmap e119: 8 total, 8 up, 8 in 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: pgmap v133: 332 pgs: 40 unknown, 292 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2203046161' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi045-74470-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: from='client.47716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi045-74490-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:30 smithi045 ceph-mon[37795]: osdmap e119: 8 total, 8 up, 8 in 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: osdmap e120: 8 total, 8 up, 8 in 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: osdmap e120: 8 total, 8 up, 8 in 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: osdmap e120: 8 total, 8 up, 8 in 2024-04-14T15:13:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:13:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: pgmap v136: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-04-14T15:13:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:13:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: osdmap e121: 8 total, 8 up, 8 in 2024-04-14T15:13:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:32.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: pgmap v136: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: osdmap e121: 8 total, 8 up, 8 in 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: pgmap v136: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: osdmap e121: 8 total, 8 up, 8 in 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]: dispatch 2024-04-14T15:13:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]': finished 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-mon[33150]: osdmap e122: 8 total, 8 up, 8 in 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]': finished 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[29624]: osdmap e122: 8 total, 8 up, 8 in 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1784550275' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi045-74470-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-13", "mode": "writeback"}]': finished 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/770198635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi045-74490-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:33.408 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:33 smithi045 ceph-mon[37795]: osdmap e122: 8 total, 8 up, 8 in 2024-04-14T15:13:33.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:34.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: pgmap v139: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-04-14T15:13:34.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:34.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:34.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:34.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:34.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: 172.9 scrub starts 2024-04-14T15:13:34.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: 172.9 scrub ok 2024-04-14T15:13:34.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:34 smithi071 ceph-mon[33150]: osdmap e123: 8 total, 8 up, 8 in 2024-04-14T15:13:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: pgmap v139: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-04-14T15:13:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: 172.9 scrub starts 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: 172.9 scrub ok 2024-04-14T15:13:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[29624]: osdmap e123: 8 total, 8 up, 8 in 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: pgmap v139: 428 pgs: 96 unknown, 8 creating+peering, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:34.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: 172.9 scrub starts 2024-04-14T15:13:34.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: 172.9 scrub ok 2024-04-14T15:13:34.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:34 smithi045 ceph-mon[37795]: osdmap e123: 8 total, 8 up, 8 in 2024-04-14T15:13:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.0 scrub starts 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.0 scrub ok 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.4 scrub starts 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.4 scrub ok 2024-04-14T15:13:35.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.8 scrub starts 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.7 scrub starts 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.8 scrub ok 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.7 scrub ok 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.3 scrub starts 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: 172.3 scrub ok 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: osdmap e124: 8 total, 8 up, 8 in 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3159714843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:35 smithi071 ceph-mon[33150]: from='client.56333 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.0 scrub starts 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.0 scrub ok 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.4 scrub starts 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.4 scrub ok 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.8 scrub starts 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.7 scrub starts 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.8 scrub ok 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.7 scrub ok 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.3 scrub starts 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: 172.3 scrub ok 2024-04-14T15:13:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: osdmap e124: 8 total, 8 up, 8 in 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3159714843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[29624]: from='client.56333 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-04-14T15:13:35.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.0 scrub starts 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.0 scrub ok 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.4 scrub starts 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.4 scrub ok 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.8 scrub starts 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.7 scrub starts 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.8 scrub ok 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.7 scrub ok 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.3 scrub starts 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: 172.3 scrub ok 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:35.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: osdmap e124: 8 total, 8 up, 8 in 2024-04-14T15:13:35.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3159714843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:35.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:35 smithi045 ceph-mon[37795]: from='client.56333 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: pgmap v142: 332 pgs: 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: 172.5 scrub starts 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: 172.5 scrub ok 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: 172.2 scrub starts 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: 172.2 scrub ok 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.56333 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: osdmap e125: 8 total, 8 up, 8 in 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:36 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: pgmap v142: 332 pgs: 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: 172.5 scrub starts 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: 172.5 scrub ok 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: 172.2 scrub starts 2024-04-14T15:13:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: 172.2 scrub ok 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.56333 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: osdmap e125: 8 total, 8 up, 8 in 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: pgmap v142: 332 pgs: 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: 172.5 scrub starts 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: 172.5 scrub ok 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: 172.2 scrub starts 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: 172.2 scrub ok 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2194845981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi045-74490-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.56333 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi045-74470-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:36.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: osdmap e125: 8 total, 8 up, 8 in 2024-04-14T15:13:36.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:36.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:37.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: 172.1 scrub starts 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: 172.1 scrub ok 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: osdmap e126: 8 total, 8 up, 8 in 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: 172.1 scrub starts 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: 172.1 scrub ok 2024-04-14T15:13:37.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: osdmap e126: 8 total, 8 up, 8 in 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:37 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: 172.1 scrub starts 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: 172.1 scrub ok 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: osdmap e126: 8 total, 8 up, 8 in 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:37 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]: dispatch 2024-04-14T15:13:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: pgmap v145: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: 172.6 scrub starts 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: 172.6 scrub ok 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: osdmap e127: 8 total, 8 up, 8 in 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4202682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[29624]: from='client.56342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: pgmap v145: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: 172.6 scrub starts 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: 172.6 scrub ok 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: osdmap e127: 8 total, 8 up, 8 in 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4202682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:38 smithi045 ceph-mon[37795]: from='client.56342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: pgmap v145: 428 pgs: 96 unknown, 332 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 53 KiB/s rd, 56 KiB/s wr, 120 op/s 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: 172.6 scrub starts 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: 172.6 scrub ok 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pg_num","val":"11"}]': finished 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: osdmap e127: 8 total, 8 up, 8 in 2024-04-14T15:13:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]: dispatch 2024-04-14T15:13:38.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:38.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4202682890' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:38.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:38 smithi071 ceph-mon[33150]: from='client.56342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:39.005 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: Running main() from gmock_main.cc 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] Global test environment set-up. 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2006 ms) 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2008 ms) 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2028 ms) 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2005 ms) 2024-04-14T15:13:39.006 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8047 ms total) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4150 ms) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (3920 ms) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5057 ms) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13127 ms total) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2994 ms) 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-04-14T15:13:39.007 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2025 ms) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2021 ms) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2008 ms) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9048 ms total) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4014 ms) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4010 ms) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8024 ms total) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [----------] Global test environment tear-down 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (55489 ms total) 2024-04-14T15:13:39.008 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: pgmap v148: 388 pgs: 7 creating+peering, 57 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: from='client.56342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: osdmap e128: 8 total, 8 up, 8 in 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:40 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: pgmap v148: 388 pgs: 7 creating+peering, 57 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-04-14T15:13:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: from='client.56342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: osdmap e128: 8 total, 8 up, 8 in 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: pgmap v148: 388 pgs: 7 creating+peering, 57 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/562354473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi045-74775-15"}]': finished 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3733002235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi045-74470-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: from='client.56342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi045-74490-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: osdmap e128: 8 total, 8 up, 8 in 2024-04-14T15:13:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:40.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:40 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:41 smithi071 ceph-mon[33150]: osdmap e129: 8 total, 8 up, 8 in 2024-04-14T15:13:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1490401205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:41 smithi071 ceph-mon[33150]: from='client.47752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[37795]: osdmap e129: 8 total, 8 up, 8 in 2024-04-14T15:13:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1490401205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[37795]: from='client.47752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[29624]: osdmap e129: 8 total, 8 up, 8 in 2024-04-14T15:13:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1490401205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[29624]: from='client.47752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: pgmap v151: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 5 op/s 2024-04-14T15:13:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.47752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: osdmap e130: 8 total, 8 up, 8 in 2024-04-14T15:13:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/777124940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.56366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1245591691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:42 smithi071 ceph-mon[33150]: from='client.47761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: pgmap v151: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 5 op/s 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.47752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: osdmap e130: 8 total, 8 up, 8 in 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/777124940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.56366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1245591691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[29624]: from='client.47761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: pgmap v151: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.2 KiB/s rd, 5 op/s 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.47752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74876-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: osdmap e130: 8 total, 8 up, 8 in 2024-04-14T15:13:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/777124940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.56366 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1245591691' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:42 smithi045 ceph-mon[37795]: from='client.47761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='client.56366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='client.47761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: osdmap e131: 8 total, 8 up, 8 in 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='client.56366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='client.47761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: osdmap e131: 8 total, 8 up, 8 in 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='client.56366 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi045-74490-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='client.47761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74470-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: osdmap e131: 8 total, 8 up, 8 in 2024-04-14T15:13:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:13:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:43.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:44 smithi071 ceph-mon[33150]: pgmap v154: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:13:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:44 smithi071 ceph-mon[33150]: osdmap e132: 8 total, 8 up, 8 in 2024-04-14T15:13:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[29624]: pgmap v154: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:13:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[29624]: osdmap e132: 8 total, 8 up, 8 in 2024-04-14T15:13:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[37795]: pgmap v154: 420 pgs: 128 unknown, 292 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 4.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:13:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[37795]: osdmap e132: 8 total, 8 up, 8 in 2024-04-14T15:13:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:45.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: osdmap e133: 8 total, 8 up, 8 in 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2743994999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.56396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2083243339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: osdmap e133: 8 total, 8 up, 8 in 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2743994999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.56396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2083243339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:45.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: osdmap e133: 8 total, 8 up, 8 in 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2743994999' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.56396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2083243339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:45.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:45.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:45.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: pgmap v157: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: from='client.56396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: osdmap e134: 8 total, 8 up, 8 in 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:46.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: pgmap v157: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:46.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: from='client.56396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: osdmap e134: 8 total, 8 up, 8 in 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: pgmap v157: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: from='client.56396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi045-74470-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi045-74490-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: osdmap e134: 8 total, 8 up, 8 in 2024-04-14T15:13:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: osdmap e135: 8 total, 8 up, 8 in 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: osdmap e136: 8 total, 8 up, 8 in 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/414620269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.47818 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2462017629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:47 smithi071 ceph-mon[33150]: from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: osdmap e135: 8 total, 8 up, 8 in 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: osdmap e136: 8 total, 8 up, 8 in 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/414620269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.47818 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2462017629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[29624]: from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: osdmap e135: 8 total, 8 up, 8 in 2024-04-14T15:13:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi045-74876-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: osdmap e136: 8 total, 8 up, 8 in 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/414620269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.47818 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2462017629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:47 smithi045 ceph-mon[37795]: from='client.47815 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: pgmap v160: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.47818 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: osdmap e137: 8 total, 8 up, 8 in 2024-04-14T15:13:48.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: pgmap v160: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.47818 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: osdmap e137: 8 total, 8 up, 8 in 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: pgmap v160: 324 pgs: 324 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]: dispatch 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.47818 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi045-74470-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.47815 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.34712 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi045-74598-1","var":"pgp_num","val":"11"}]': finished 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: osdmap e137: 8 total, 8 up, 8 in 2024-04-14T15:13:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2250092577' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: osdmap e138: 8 total, 8 up, 8 in 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2"}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:49 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: osdmap e138: 8 total, 8 up, 8 in 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2"}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi045-74598-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: osdmap e138: 8 total, 8 up, 8 in 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2"}]: dispatch 2024-04-14T15:13:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:49.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:49 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: pgmap v163: 396 pgs: 58 unknown, 338 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: osdmap e139: 8 total, 8 up, 8 in 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/905419537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47833 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3193021338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:50 smithi071 ceph-mon[33150]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: pgmap v163: 396 pgs: 58 unknown, 338 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: osdmap e139: 8 total, 8 up, 8 in 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/905419537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47833 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3193021338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[37795]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: pgmap v163: 396 pgs: 58 unknown, 338 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi045-74699-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi045-74598-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: osdmap e139: 8 total, 8 up, 8 in 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/905419537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47833 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3193021338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:50.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:50 smithi045 ceph-mon[29624]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:52.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-04-14T15:13:52.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-04-14T15:13:52.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-04-14T15:13:52.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: seed 74876 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2004 ms) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2009 ms) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2029 ms) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2001 ms) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3697 ms) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (11740 ms total) 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:13:52.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4155 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3899 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5976 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4021 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2063 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2778-ga971fd5f/rpm/el9/BUILD/ceph-19.0.0-2778-ga971fd5f/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-04-14T15:13:52.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi045-74876-7 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: waiting for snaps to purge 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17962 ms) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38076 ms total) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5078 ms) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5087 ms total) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2927 ms) 2024-04-14T15:13:52.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-04-14T15:13:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: pgmap v166: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:13:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: from='client.47833 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:13:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: osdmap e140: 8 total, 8 up, 8 in 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: pgmap v166: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: from='client.47833 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: osdmap e140: 8 total, 8 up, 8 in 2024-04-14T15:13:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:51 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: pgmap v166: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: from='client.47833 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi045-74470-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi045-74490-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: osdmap e140: 8 total, 8 up, 8 in 2024-04-14T15:13:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:51 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-04-14T15:13:53.069 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapsexpected=0 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=7 expected=7 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=12 expected=12 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=4 expected=4 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-04-14T15:13:53.070 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=3 expected=3 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=1 expected=1 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=14 expected=14 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=10 expected=10 2024-04-14T15:13:53.071 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : seek to 14:566253c9:::13:head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: : entry=13 expected=13 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (228 ms) 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (50970 ms) 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (11058 ms) 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 7 tests from LibRadosList (62908 ms total) 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 2024-04-14T15:13:53.072 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1028 ms) 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo1,foo2,foo3 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo1 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo2 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo3 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo1,foo4,foo5 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo4 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo5 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo1 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset foo6,foo7 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo7 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: foo6 2024-04-14T15:13:53.073 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo4 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo5 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns2:foo7 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns2:foo6 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: ns1:foo1 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo1 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo2 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: :foo3 2024-04-14T15:13:53.074 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (33 ms) 2024-04-14T15:13:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:13:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-mon[33150]: osdmap e141: 8 total, 8 up, 8 in 2024-04-14T15:13:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:53.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[29624]: osdmap e141: 8 total, 8 up, 8 in 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi045-74598-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[37795]: osdmap e141: 8 total, 8 up, 8 in 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181074549' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:53 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]: dispatch 2024-04-14T15:13:53.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:13:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:13:54.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosListEC.Lis api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-04-14T15:13:54.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-04-14T15:13:54.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: seed 74699 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-04-14T15:13:54.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (418 ms) 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (14 ms) 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (9 ms) 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (13 ms) 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (3 ms) 2024-04-14T15:13:54.012 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-04-14T15:13:54.013 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (12 ms) 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (10 ms) 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: osd_max_attr_size = 0 2024-04-14T15:13:54.014 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1760 ms) 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (368 ms) 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: done waiting, doing copies 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: waiting for final deep scrubs... 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: done waiting 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60694 ms) 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-04-14T15:13:54.015 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-04-14T15:13:54.016 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-04-14T15:13:54.016 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-04-14T15:13:54.016 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-04-14T15:13:54.016 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4450 ms) 2024-04-14T15:13:54.016 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-04-14T15:13:54.017 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-04-14T15:13:54.017 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-04-14T15:13:54.017 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-04-14T15:13:54.017 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-04-14T15:13:54.017 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: pgmap v169: 308 pgs: 8 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: osdmap e142: 8 total, 8 up, 8 in 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1013131140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2214660681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: from='client.56441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:54 smithi071 ceph-mon[33150]: from='client.56444 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: pgmap v169: 308 pgs: 8 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:13:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: osdmap e142: 8 total, 8 up, 8 in 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1013131140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2214660681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: from='client.56441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[29624]: from='client.56444 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: pgmap v169: 308 pgs: 8 unknown, 300 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: from='client.34709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi045-74699-1","app":"app1","key":"key1"}]': finished 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: osdmap e142: 8 total, 8 up, 8 in 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1013131140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2214660681' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: from='client.56441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:54 smithi045 ceph-mon[37795]: from='client.56444 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.56441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.56444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: osdmap e143: 8 total, 8 up, 8 in 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: osdmap e144: 8 total, 8 up, 8 in 2024-04-14T15:13:55.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.56441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.56444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: osdmap e143: 8 total, 8 up, 8 in 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: osdmap e144: 8 total, 8 up, 8 in 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.56441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi045-74470-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.56444 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi045-74490-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: osdmap e143: 8 total, 8 up, 8 in 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:13:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:13:55.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:55.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74699-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:13:55.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: osdmap e144: 8 total, 8 up, 8 in 2024-04-14T15:13:55.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:13:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:56 smithi071 ceph-mon[33150]: pgmap v172: 332 pgs: 64 creating+peering, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:56 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:56 smithi071 ceph-mon[33150]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[29624]: pgmap v172: 332 pgs: 64 creating+peering, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[29624]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[37795]: pgmap v172: 332 pgs: 64 creating+peering, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3037067871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]: dispatch 2024-04-14T15:13:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-mon[37795]: from='client.47836 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi045-74598-2"}]': finished 2024-04-14T15:13:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:13:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:13:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: osdmap e145: 8 total, 8 up, 8 in 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1331723799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.47860 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.47860 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: osdmap e146: 8 total, 8 up, 8 in 2024-04-14T15:13:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: osdmap e145: 8 total, 8 up, 8 in 2024-04-14T15:13:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1331723799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.47860 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.47860 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: osdmap e146: 8 total, 8 up, 8 in 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: osdmap e145: 8 total, 8 up, 8 in 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1331723799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.47860 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74699-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:13:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.47860 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi045-74470-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2249911970' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi045-74490-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: osdmap e146: 8 total, 8 up, 8 in 2024-04-14T15:13:57.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP snapshots: Running main() from gmock_main.cc 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] Global test environment set-up. 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (4389 ms) 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5036 ms) 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3691 ms) 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5017 ms) 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-04-14T15:13:58.027 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7055 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (25188 ms total) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4967 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6034 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7946 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5997 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4007 ms) 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-04-14T15:13:58.028 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi045-75916-11. 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: Waiting for snaps to purge. 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20023 ms) 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48974 ms total) 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [----------] Global test environment tear-down 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (74162 ms total) 2024-04-14T15:13:58.029 INFO:tasks.workunit.client.0.smithi045.stdout: snapshots: [ PASSED ] 11 tests. 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: pgmap v175: 332 pgs: 64 unknown, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.47869 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.47869 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: osdmap e147: 8 total, 8 up, 8 in 2024-04-14T15:13:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: pgmap v175: 332 pgs: 64 unknown, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.47869 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.47869 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: osdmap e147: 8 total, 8 up, 8 in 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: pgmap v175: 332 pgs: 64 unknown, 268 active+clean; 590 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.47869 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:13:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.47869 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74598-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: osdmap e147: 8 total, 8 up, 8 in 2024-04-14T15:13:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]: dispatch 2024-04-14T15:13:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout:tObjectsStart 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 1 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 10 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 13 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 7 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 14 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 0 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 15 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 11 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 5 0 2024-04-14T15:13:59.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 8 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 6 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 3 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 4 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 12 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 9 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 2 0 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (44 ms) 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 3 tests from LibRadosListEC (1105 ms total) 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3005 ms) 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] 1 test from LibRadosListNP (3005 ms total) 2024-04-14T15:13:59.029 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: 2024-04-14T15:13:59.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [----------] Global test environment tear-down 2024-04-14T15:13:59.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [==========] 11 tests from 3 test suites ran. (75534 ms total) 2024-04-14T15:13:59.030 INFO:tasks.workunit.client.0.smithi045.stdout: api_list: [ PASSED ] 11 tests. 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.47869 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.47869 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: osdmap e148: 8 total, 8 up, 8 in 2024-04-14T15:13:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3711531463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/277107472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.47878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:13:59 smithi071 ceph-mon[33150]: from='client.56474 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.47869 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.47869 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: osdmap e148: 8 total, 8 up, 8 in 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3711531463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/277107472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.47878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[37795]: from='client.56474 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3111262754' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.47869 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/478183192' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi045-74876-16"}]': finished 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.47869 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi045-74598-3","pool2":"test-rados-api-smithi045-74598-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: osdmap e148: 8 total, 8 up, 8 in 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3711531463' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/277107472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.47878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:13:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:13:59 smithi045 ceph-mon[29624]: from='client.56474 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: pgmap v178: 300 pgs: 2 creating+peering, 64 unknown, 234 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.56474 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: osdmap e149: 8 total, 8 up, 8 in 2024-04-14T15:14:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:00 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: pgmap v178: 300 pgs: 2 creating+peering, 64 unknown, 234 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.56474 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: osdmap e149: 8 total, 8 up, 8 in 2024-04-14T15:14:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: pgmap v178: 300 pgs: 2 creating+peering, 64 unknown, 234 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74699-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi045-74490-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.56474 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi045-74470-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: osdmap e149: 8 total, 8 up, 8 in 2024-04-14T15:14:00.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:00 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:02.298 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: pgmap v181: 332 pgs: 32 creating+peering, 64 unknown, 236 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: osdmap e150: 8 total, 8 up, 8 in 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: pgmap v181: 332 pgs: 32 creating+peering, 64 unknown, 236 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: osdmap e150: 8 total, 8 up, 8 in 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:02 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: pgmap v181: 332 pgs: 32 creating+peering, 64 unknown, 236 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: osdmap e150: 8 total, 8 up, 8 in 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:02 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: osdmap e151: 8 total, 8 up, 8 in 2024-04-14T15:14:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/630823385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-mon[33150]: from='client.47899 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: osdmap e151: 8 total, 8 up, 8 in 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/630823385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[37795]: from='client.47899 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi045-74470-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: osdmap e151: 8 total, 8 up, 8 in 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/630823385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:03 smithi045 ceph-mon[29624]: from='client.47899 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:03.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:04 smithi071 ceph-mon[33150]: pgmap v184: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:04 smithi071 ceph-mon[33150]: from='client.47899 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:04 smithi071 ceph-mon[33150]: osdmap e152: 8 total, 8 up, 8 in 2024-04-14T15:14:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[37795]: pgmap v184: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[37795]: from='client.47899 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[37795]: osdmap e152: 8 total, 8 up, 8 in 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[29624]: pgmap v184: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[29624]: from='client.47899 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi045-74490-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[29624]: osdmap e152: 8 total, 8 up, 8 in 2024-04-14T15:14:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]: dispatch 2024-04-14T15:14:05.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:05.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:05 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:05.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:05.360 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:05 smithi071 ceph-mon[33150]: osdmap e153: 8 total, 8 up, 8 in 2024-04-14T15:14:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:05.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[37795]: osdmap e153: 8 total, 8 up, 8 in 2024-04-14T15:14:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi045-74470-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/784524459' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74699-24"}]': finished 2024-04-14T15:14:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:05 smithi045 ceph-mon[29624]: osdmap e153: 8 total, 8 up, 8 in 2024-04-14T15:14:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: pgmap v187: 244 pgs: 8 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: osdmap e154: 8 total, 8 up, 8 in 2024-04-14T15:14:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3922804616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3569370793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:06 smithi071 ceph-mon[33150]: from='client.47908 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: pgmap v187: 244 pgs: 8 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: osdmap e154: 8 total, 8 up, 8 in 2024-04-14T15:14:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3922804616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3569370793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[29624]: from='client.47908 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: pgmap v187: 244 pgs: 8 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: osdmap e154: 8 total, 8 up, 8 in 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3922804616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3569370793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-mon[37795]: from='client.47908 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:07 smithi071 ceph-mon[33150]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:07 smithi071 ceph-mon[33150]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:07 smithi071 ceph-mon[33150]: osdmap e155: 8 total, 8 up, 8 in 2024-04-14T15:14:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:07 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[37795]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[37795]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[37795]: osdmap e155: 8 total, 8 up, 8 in 2024-04-14T15:14:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[29624]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi045-74490-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[29624]: from='client.47908 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi045-74699-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[29624]: osdmap e155: 8 total, 8 up, 8 in 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:07 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: Running main() from gmock_main.cc 2024-04-14T15:14:08.081 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] Global test environment set-up. 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.TooBig (2835 ms) 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3004 ms) 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3030 ms) 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2694 ms) 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3042 ms) 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-04-14T15:14:08.082 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2965 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3037 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3000 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2995 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3007 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2972 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3037 ms) 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-04-14T15:14:08.083 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3017 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2934 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.Flush (3009 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2999 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3005 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3001 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3009 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3042 ms) 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-04-14T15:14:08.084 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3023 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3000 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2941 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3003 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3013 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3008 ms) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] 26 tests from LibRadosAio (77622 ms total) 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-04-14T15:14:08.085 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: pgmap v190: 300 pgs: 64 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: osdmap e156: 8 total, 8 up, 8 in 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: pgmap v190: 300 pgs: 64 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: osdmap e156: 8 total, 8 up, 8 in 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: pgmap v190: 300 pgs: 64 unknown, 236 active+clean; 598 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: osdmap e156: 8 total, 8 up, 8 in 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1333879431' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]: dispatch 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:08.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: osdmap e157: 8 total, 8 up, 8 in 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2363546777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.47917 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: osdmap e157: 8 total, 8 up, 8 in 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2363546777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.47917 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.56492 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi045-74470-27"}]': finished 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:09.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: osdmap e157: 8 total, 8 up, 8 in 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2363546777' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.47917 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:09.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: pgmap v193: 300 pgs: 17 creating+peering, 47 unknown, 236 active+clean; 596 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: from='client.47917 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: osdmap e158: 8 total, 8 up, 8 in 2024-04-14T15:14:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: pgmap v193: 300 pgs: 17 creating+peering, 47 unknown, 236 active+clean; 596 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: from='client.47917 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: osdmap e158: 8 total, 8 up, 8 in 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: pgmap v193: 300 pgs: 17 creating+peering, 47 unknown, 236 active+clean; 596 KiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-13"}]': finished 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: from='client.47917 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi045-74699-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424820359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi045-74490-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi045-74470-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: osdmap e158: 8 total, 8 up, 8 in 2024-04-14T15:14:10.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: osdmap e159: 8 total, 8 up, 8 in 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: osdmap e160: 8 total, 8 up, 8 in 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3922476152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[29624]: from='client.47935 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: osdmap e159: 8 total, 8 up, 8 in 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: osdmap e160: 8 total, 8 up, 8 in 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3922476152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:11 smithi045 ceph-mon[37795]: from='client.47935 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: osdmap e159: 8 total, 8 up, 8 in 2024-04-14T15:14:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi045-74470-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: osdmap e160: 8 total, 8 up, 8 in 2024-04-14T15:14:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3922476152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:11.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:11 smithi071 ceph-mon[33150]: from='client.47935 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[37795]: pgmap v196: 204 pgs: 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[37795]: from='client.47935 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[37795]: osdmap e161: 8 total, 8 up, 8 in 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[29624]: pgmap v196: 204 pgs: 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[29624]: from='client.47935 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:12 smithi045 ceph-mon[29624]: osdmap e161: 8 total, 8 up, 8 in 2024-04-14T15:14:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:12 smithi071 ceph-mon[33150]: pgmap v196: 204 pgs: 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-04-14T15:14:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2750380497' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi045-74490-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:12 smithi071 ceph-mon[33150]: from='client.47935 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi045-74699-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:12 smithi071 ceph-mon[33150]: osdmap e161: 8 total, 8 up, 8 in 2024-04-14T15:14:13.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: pgmap v199: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: osdmap e162: 8 total, 8 up, 8 in 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:13.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:13 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: pgmap v199: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: osdmap e162: 8 total, 8 up, 8 in 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: pgmap v199: 308 pgs: 104 unknown, 204 active+clean; 608 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: osdmap e162: 8 total, 8 up, 8 in 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:13.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:13.999 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:13 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.096 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: osdmap e163: 8 total, 8 up, 8 in 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3408924554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.56543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.097 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.56543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: osdmap e164: 8 total, 8 up, 8 in 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.098 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:15 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: osdmap e163: 8 total, 8 up, 8 in 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3408924554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.56543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.56543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: osdmap e163: 8 total, 8 up, 8 in 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3408924554' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.56543 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/931187055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi045-74470-28"}]': finished 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.56543 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi045-74490-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:15.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: osdmap e164: 8 total, 8 up, 8 in 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: osdmap e164: 8 total, 8 up, 8 in 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:15.501 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:15 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: pgmap v202: 268 pgs: 32 unknown, 236 active+clean; 608 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:14:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:14:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:14:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]': finished 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: osdmap e165: 8 total, 8 up, 8 in 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:16 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: pgmap v202: 268 pgs: 32 unknown, 236 active+clean; 608 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]': finished 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: osdmap e165: 8 total, 8 up, 8 in 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: pgmap v202: 268 pgs: 32 unknown, 236 active+clean; 608 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi045-74699-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-15", "mode": "writeback"}]': finished 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi045-74470-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: osdmap e165: 8 total, 8 up, 8 in 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/425568187' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]: dispatch 2024-04-14T15:14:16.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: hotsECPP.SnapRemovePP (2010 ms) 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2008 ms) 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2016 ms) 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8961 ms total) 2024-04-14T15:14:17.010 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4049 ms) 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (4006 ms) 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4005 ms) 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12060 ms total) 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (93448 ms total) 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-04-14T15:14:17.011 INFO:tasks.workunit.client.0.smithi045.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-04-14T15:14:18.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: pgmap v205: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: osdmap e166: 8 total, 8 up, 8 in 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1522906450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: pgmap v205: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: osdmap e166: 8 total, 8 up, 8 in 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1522906450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:18.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:17 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: pgmap v205: 236 pgs: 8 unknown, 228 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: from='client.47887 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi045-74876-21"}]': finished 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: osdmap e166: 8 total, 8 up, 8 in 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1522906450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: from='client.56555 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:17 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: osdmap e167: 8 total, 8 up, 8 in 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:19 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: osdmap e167: 8 total, 8 up, 8 in 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi045-74470-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.56555 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi045-74490-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: osdmap e167: 8 total, 8 up, 8 in 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:19 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: (67779 ms total) 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (112 ms) 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (113 ms total) 2024-04-14T15:14:20.025 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (47 ms) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (8 ms) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (55 ms total) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (37 ms) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (40 ms total) 2024-04-14T15:14:20.026 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (53 ms) 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (2 ms) 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (55 ms total) 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-04-14T15:14:20.027 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1031 ms) 2024-04-14T15:14:20.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1031 ms total) 2024-04-14T15:14:20.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: 2024-04-14T15:14:20.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-04-14T15:14:20.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (96521 ms total) 2024-04-14T15:14:20.028 INFO:tasks.workunit.client.0.smithi045.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-04-14T15:14:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: pgmap v208: 268 pgs: 1 creating+peering, 30 unknown, 237 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: osdmap e168: 8 total, 8 up, 8 in 2024-04-14T15:14:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:20 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: pgmap v208: 268 pgs: 1 creating+peering, 30 unknown, 237 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: osdmap e168: 8 total, 8 up, 8 in 2024-04-14T15:14:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: pgmap v208: 268 pgs: 1 creating+peering, 30 unknown, 237 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-15"}]': finished 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: osdmap e168: 8 total, 8 up, 8 in 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/12345046' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]: dispatch 2024-04-14T15:14:20.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:20 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: osdmap e169: 8 total, 8 up, 8 in 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1957999704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: osdmap e170: 8 total, 8 up, 8 in 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:21.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: osdmap e169: 8 total, 8 up, 8 in 2024-04-14T15:14:21.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1957999704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: osdmap e170: 8 total, 8 up, 8 in 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.47947 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi045-74699-36"}]': finished 2024-04-14T15:14:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: osdmap e169: 8 total, 8 up, 8 in 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1957999704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi045-74490-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: osdmap e170: 8 total, 8 up, 8 in 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/106719563' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:22 smithi071 ceph-mon[33150]: pgmap v211: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:22 smithi071 ceph-mon[33150]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:22 smithi071 ceph-mon[33150]: osdmap e171: 8 total, 8 up, 8 in 2024-04-14T15:14:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[37795]: pgmap v211: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[37795]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[37795]: osdmap e171: 8 total, 8 up, 8 in 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[29624]: pgmap v211: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[29624]: from='client.47953 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi045-74470-29"}]': finished 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:22 smithi045 ceph-mon[29624]: osdmap e171: 8 total, 8 up, 8 in 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: osdmap e172: 8 total, 8 up, 8 in 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/701116647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.47974 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: osdmap e172: 8 total, 8 up, 8 in 2024-04-14T15:14:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/701116647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.47974 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi045-74470-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: osdmap e172: 8 total, 8 up, 8 in 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/701116647' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.47974 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:23.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: pgmap v214: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: from='client.47974 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: osdmap e173: 8 total, 8 up, 8 in 2024-04-14T15:14:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: pgmap v214: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: from='client.47974 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: osdmap e173: 8 total, 8 up, 8 in 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: pgmap v214: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: from='client.47974 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi045-74490-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: osdmap e173: 8 total, 8 up, 8 in 2024-04-14T15:14:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]: dispatch 2024-04-14T15:14:25.000 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-04-14T15:14:25.000 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-04-14T15:14:25.000 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-04-14T15:14:25.000 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-04-14T15:14:25.000 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (2840 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (20797 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (5950 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2990 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3033 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3035 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2921 ms) 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-04-14T15:14:25.001 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3009 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2998 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3004 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3000 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3004 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3048 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3023 ms) 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-04-14T15:14:25.002 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3000 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2941 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3005 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3012 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3008 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3010 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3030 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3022 ms) 2024-04-14T15:14:25.003 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2997 ms) 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2911 ms) 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3010 ms) 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3005 ms) 2024-04-14T15:14:25.004 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]': finished 2024-04-14T15:14:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:25 smithi071 ceph-mon[33150]: osdmap e174: 8 total, 8 up, 8 in 2024-04-14T15:14:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]': finished 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[37795]: osdmap e174: 8 total, 8 up, 8 in 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi045-74470-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-17", "mode": "writeback"}]': finished 2024-04-14T15:14:25.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:25 smithi045 ceph-mon[29624]: osdmap e174: 8 total, 8 up, 8 in 2024-04-14T15:14:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: osdmap e175: 8 total, 8 up, 8 in 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/236825736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:26 smithi071 ceph-mon[33150]: from='client.47983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: osdmap e175: 8 total, 8 up, 8 in 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/236825736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[29624]: from='client.47983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: pgmap v217: 260 pgs: 32 creating+peering, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: osdmap e175: 8 total, 8 up, 8 in 2024-04-14T15:14:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]: dispatch 2024-04-14T15:14:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/236825736' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-mon[37795]: from='client.47983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: from='client.47983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: osdmap e176: 8 total, 8 up, 8 in 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:27 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: from='client.47983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: osdmap e176: 8 total, 8 up, 8 in 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-17"}]': finished 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: from='client.47983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi045-74490-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: osdmap e176: 8 total, 8 up, 8 in 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:27 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: pgmap v220: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: osdmap e177: 8 total, 8 up, 8 in 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:28 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: pgmap v220: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: osdmap e177: 8 total, 8 up, 8 in 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: pgmap v220: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: osdmap e177: 8 total, 8 up, 8 in 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1319286767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:28 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]: dispatch 2024-04-14T15:14:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: pgmap v223: 196 pgs: 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: osdmap e178: 8 total, 8 up, 8 in 2024-04-14T15:14:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/305707227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.47989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:30 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: pgmap v223: 196 pgs: 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: osdmap e178: 8 total, 8 up, 8 in 2024-04-14T15:14:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/305707227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.47989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: pgmap v223: 196 pgs: 196 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.47977 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi045-74470-30"}]': finished 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: osdmap e178: 8 total, 8 up, 8 in 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/305707227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.47989 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:30.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:30.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:30 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.47989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: osdmap e179: 8 total, 8 up, 8 in 2024-04-14T15:14:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:31.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.47989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: osdmap e179: 8 total, 8 up, 8 in 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.47989 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi045-74490-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi045-74470-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: osdmap e179: 8 total, 8 up, 8 in 2024-04-14T15:14:31.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:31.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:31.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:32 smithi071 ceph-mon[33150]: pgmap v226: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:32 smithi071 ceph-mon[33150]: osdmap e180: 8 total, 8 up, 8 in 2024-04-14T15:14:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[37795]: pgmap v226: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[37795]: osdmap e180: 8 total, 8 up, 8 in 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[29624]: pgmap v226: 260 pgs: 64 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[29624]: osdmap e180: 8 total, 8 up, 8 in 2024-04-14T15:14:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: osdmap e181: 8 total, 8 up, 8 in 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1050001293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-mon[33150]: from='client.56597 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: osdmap e181: 8 total, 8 up, 8 in 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1050001293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[29624]: from='client.56597 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi045-74470-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: osdmap e181: 8 total, 8 up, 8 in 2024-04-14T15:14:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1050001293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]: dispatch 2024-04-14T15:14:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:33 smithi045 ceph-mon[37795]: from='client.56597 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:33.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: pgmap v229: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]': finished 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: from='client.56597 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: osdmap e182: 8 total, 8 up, 8 in 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: pgmap v229: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:14:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]': finished 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: from='client.56597 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: osdmap e182: 8 total, 8 up, 8 in 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: pgmap v229: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-19", "mode": "writeback"}]': finished 2024-04-14T15:14:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: from='client.56597 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi045-74490-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:34.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: osdmap e182: 8 total, 8 up, 8 in 2024-04-14T15:14:34.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:34.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: osdmap e183: 8 total, 8 up, 8 in 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:35 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: osdmap e183: 8 total, 8 up, 8 in 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: osdmap e183: 8 total, 8 up, 8 in 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]: dispatch 2024-04-14T15:14:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:35 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: pgmap v232: 228 pgs: 228 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: osdmap e184: 8 total, 8 up, 8 in 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.56603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:36 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: pgmap v232: 228 pgs: 228 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:14:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: osdmap e184: 8 total, 8 up, 8 in 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.56603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: pgmap v232: 228 pgs: 228 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-19"}]': finished 2024-04-14T15:14:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: osdmap e184: 8 total, 8 up, 8 in 2024-04-14T15:14:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.56603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4231204876' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]: dispatch 2024-04-14T15:14:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.56603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: osdmap e185: 8 total, 8 up, 8 in 2024-04-14T15:14:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "prefix": "osd pool set", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "var": "eio", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "val": "true" 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: }]: dispatch 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: from='client.56603 ' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "prefix": "osd pool set", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "var": "eio", 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: "val": "true" 2024-04-14T15:14:37.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:37 smithi071 ceph-mon[33150]: }]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.56603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: osdmap e185: 8 total, 8 up, 8 in 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "prefix": "osd pool set", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "var": "eio", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "val": "true" 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: }]: dispatch 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: from='client.56603 ' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "prefix": "osd pool set", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "var": "eio", 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: "val": "true" 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[29624]: }]: dispatch 2024-04-14T15:14:37.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.56603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi045-74490-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.56591 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi045-74470-31"}]': finished 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: osdmap e185: 8 total, 8 up, 8 in 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3369683005' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "prefix": "osd pool set", 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "var": "eio", 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "val": "true" 2024-04-14T15:14:37.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: }]: dispatch 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: from='client.56603 ' entity='client.admin' cmd=[{ 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "prefix": "osd pool set", 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "var": "eio", 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: "val": "true" 2024-04-14T15:14:37.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:37 smithi045 ceph-mon[37795]: }]: dispatch 2024-04-14T15:14:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: pgmap v235: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.56603 ' entity='client.admin' cmd='[{ 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: "prefix": "osd pool set", 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: "var": "eio", 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: "val": "true" 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: }]': finished 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: osdmap e186: 8 total, 8 up, 8 in 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:38.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:38 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: pgmap v235: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.56603 ' entity='client.admin' cmd='[{ 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: "prefix": "osd pool set", 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: "var": "eio", 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: "val": "true" 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: }]': finished 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: osdmap e186: 8 total, 8 up, 8 in 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: pgmap v235: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi045-74470-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.56603 ' entity='client.admin' cmd='[{ 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: "prefix": "osd pool set", 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: "pool": "PoolEIOFlag_smithi045-74490-33", 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: "var": "eio", 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: "val": "true" 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: }]': finished 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: osdmap e186: 8 total, 8 up, 8 in 2024-04-14T15:14:38.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:38.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:38.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:38 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:39 smithi071 ceph-mon[33150]: osdmap e187: 8 total, 8 up, 8 in 2024-04-14T15:14:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[29624]: osdmap e187: 8 total, 8 up, 8 in 2024-04-14T15:14:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[37795]: osdmap e187: 8 total, 8 up, 8 in 2024-04-14T15:14:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: pgmap v238: 228 pgs: 17 unknown, 211 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: osdmap e188: 8 total, 8 up, 8 in 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2324774866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: from='client.48019 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:40 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: pgmap v238: 228 pgs: 17 unknown, 211 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: osdmap e188: 8 total, 8 up, 8 in 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2324774866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: from='client.48019 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: pgmap v238: 228 pgs: 17 unknown, 211 active+clean; 584 KiB data, 557 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:14:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi045-74470-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: osdmap e188: 8 total, 8 up, 8 in 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2324774866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: from='client.48019 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:40 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:41 smithi071 ceph-mon[33150]: from='client.48019 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:41.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:41 smithi071 ceph-mon[33150]: osdmap e189: 8 total, 8 up, 8 in 2024-04-14T15:14:41.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:41.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[37795]: from='client.48019 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[37795]: osdmap e189: 8 total, 8 up, 8 in 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[29624]: from='client.48019 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi045-74490-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[29624]: osdmap e189: 8 total, 8 up, 8 in 2024-04-14T15:14:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]: dispatch 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: pgmap v241: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]': finished 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: osdmap e190: 8 total, 8 up, 8 in 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: pgmap v241: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]': finished 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: osdmap e190: 8 total, 8 up, 8 in 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: pgmap v241: 268 pgs: 40 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-21", "mode": "writeback"}]': finished 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: osdmap e190: 8 total, 8 up, 8 in 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: osdmap e191: 8 total, 8 up, 8 in 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3003208022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: osdmap e191: 8 total, 8 up, 8 in 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3003208022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: osdmap e191: 8 total, 8 up, 8 in 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3003208022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4210295613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]: dispatch 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:43.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: pgmap v244: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: osdmap e192: 8 total, 8 up, 8 in 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: pgmap v244: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: osdmap e192: 8 total, 8 up, 8 in 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:44.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:44 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: pgmap v244: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 561 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-21"}]': finished 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi045-74490-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.48013 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi045-74470-32"}]': finished 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: osdmap e192: 8 total, 8 up, 8 in 2024-04-14T15:14:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:44 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: osdmap e193: 8 total, 8 up, 8 in 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[37795]: osdmap e194: 8 total, 8 up, 8 in 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: osdmap e193: 8 total, 8 up, 8 in 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:45 smithi045 ceph-mon[29624]: osdmap e194: 8 total, 8 up, 8 in 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi045-74470-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: osdmap e193: 8 total, 8 up, 8 in 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:45 smithi071 ceph-mon[33150]: osdmap e194: 8 total, 8 up, 8 in 2024-04-14T15:14:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: pgmap v247: 196 pgs: 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/460068530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[29624]: osdmap e195: 8 total, 8 up, 8 in 2024-04-14T15:14:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: pgmap v247: 196 pgs: 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/460068530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-mon[37795]: osdmap e195: 8 total, 8 up, 8 in 2024-04-14T15:14:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: pgmap v247: 196 pgs: 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/460068530' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi045-74470-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:46.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:46 smithi071 ceph-mon[33150]: osdmap e195: 8 total, 8 up, 8 in 2024-04-14T15:14:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[37795]: pgmap v250: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[37795]: osdmap e196: 8 total, 8 up, 8 in 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[29624]: pgmap v250: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[29624]: osdmap e196: 8 total, 8 up, 8 in 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:48.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:48 smithi071 ceph-mon[33150]: pgmap v250: 268 pgs: 72 unknown, 196 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:14:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:48 smithi071 ceph-mon[33150]: osdmap e196: 8 total, 8 up, 8 in 2024-04-14T15:14:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: osdmap e197: 8 total, 8 up, 8 in 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: osdmap e198: 8 total, 8 up, 8 in 2024-04-14T15:14:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: osdmap e197: 8 total, 8 up, 8 in 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:49.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:49.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: osdmap e198: 8 total, 8 up, 8 in 2024-04-14T15:14:49.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:49.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2441360359' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: osdmap e197: 8 total, 8 up, 8 in 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:49.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: osdmap e198: 8 total, 8 up, 8 in 2024-04-14T15:14:49.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:49.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2341818193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:49.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]: dispatch 2024-04-14T15:14:50.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: pgmap v253: 292 pgs: 62 unknown, 230 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/179136615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]': finished 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[37795]: osdmap e199: 8 total, 8 up, 8 in 2024-04-14T15:14:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: pgmap v253: 292 pgs: 62 unknown, 230 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/179136615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]': finished 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:50 smithi045 ceph-mon[29624]: osdmap e199: 8 total, 8 up, 8 in 2024-04-14T15:14:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: pgmap v253: 292 pgs: 62 unknown, 230 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]: dispatch 2024-04-14T15:14:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/179136615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-23", "mode": "writeback"}]': finished 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi045-74470-33"}]': finished 2024-04-14T15:14:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi045-74490-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:50.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:50 smithi071 ceph-mon[33150]: osdmap e199: 8 total, 8 up, 8 in 2024-04-14T15:14:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: osdmap e200: 8 total, 8 up, 8 in 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: osdmap e200: 8 total, 8 up, 8 in 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:51 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi045-74470-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: osdmap e200: 8 total, 8 up, 8 in 2024-04-14T15:14:51.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:51.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:51 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:52.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: pgmap v256: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: osdmap e201: 8 total, 8 up, 8 in 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: pgmap v256: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: osdmap e201: 8 total, 8 up, 8 in 2024-04-14T15:14:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: pgmap v256: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: osdmap e201: 8 total, 8 up, 8 in 2024-04-14T15:14:52.510 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]: dispatch 2024-04-14T15:14:53.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[37795]: osdmap e202: 8 total, 8 up, 8 in 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:53 smithi045 ceph-mon[29624]: osdmap e202: 8 total, 8 up, 8 in 2024-04-14T15:14:53.591 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:53 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:14:53.591 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:53 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi045-74470-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:53.591 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-23"}]': finished 2024-04-14T15:14:53.591 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:53 smithi071 ceph-mon[33150]: osdmap e202: 8 total, 8 up, 8 in 2024-04-14T15:14:53.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:14:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:14:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[37795]: pgmap v259: 260 pgs: 260 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-14T15:14:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[37795]: osdmap e203: 8 total, 8 up, 8 in 2024-04-14T15:14:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2199130665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[37795]: from='client.48055 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[29624]: pgmap v259: 260 pgs: 260 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-14T15:14:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[29624]: osdmap e203: 8 total, 8 up, 8 in 2024-04-14T15:14:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2199130665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:54 smithi045 ceph-mon[29624]: from='client.48055 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:54 smithi071 ceph-mon[33150]: pgmap v259: 260 pgs: 260 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s wr, 2 op/s 2024-04-14T15:14:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:54 smithi071 ceph-mon[33150]: osdmap e203: 8 total, 8 up, 8 in 2024-04-14T15:14:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2199130665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:54 smithi071 ceph-mon[33150]: from='client.48055 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.48055 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: osdmap e204: 8 total, 8 up, 8 in 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.48055 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: osdmap e204: 8 total, 8 up, 8 in 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.48055 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: osdmap e204: 8 total, 8 up, 8 in 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: pgmap v262: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:56.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:56.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: osdmap e205: 8 total, 8 up, 8 in 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/8822413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[37795]: from='client.56660 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: pgmap v262: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: osdmap e205: 8 total, 8 up, 8 in 2024-04-14T15:14:56.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/8822413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-mon[29624]: from='client.56660 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: pgmap v262: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1065979620' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: osdmap e205: 8 total, 8 up, 8 in 2024-04-14T15:14:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]: dispatch 2024-04-14T15:14:56.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/8822413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:56 smithi071 ceph-mon[33150]: from='client.56660 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:14:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:14:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:14:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: pgmap v265: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.56660 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: osdmap e206: 8 total, 8 up, 8 in 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: pgmap v265: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.56660 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: osdmap e206: 8 total, 8 up, 8 in 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:58.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: pgmap v265: 292 pgs: 96 unknown, 196 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:14:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.48049 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi045-74470-34"}]': finished 2024-04-14T15:14:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.56660 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:14:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: osdmap e206: 8 total, 8 up, 8 in 2024-04-14T15:14:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:14:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:14:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: osdmap e207: 8 total, 8 up, 8 in 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: osdmap e207: 8 total, 8 up, 8 in 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:59.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:14:59 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:14:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi045-74470-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:14:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: osdmap e207: 8 total, 8 up, 8 in 2024-04-14T15:14:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:14:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:14:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:14:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:14:59 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: pgmap v268: 324 pgs: 14 creating+peering, 74 unknown, 236 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: osdmap e208: 8 total, 8 up, 8 in 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]': finished 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: osdmap e209: 8 total, 8 up, 8 in 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2715480966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: pgmap v268: 324 pgs: 14 creating+peering, 74 unknown, 236 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: osdmap e208: 8 total, 8 up, 8 in 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]': finished 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: osdmap e209: 8 total, 8 up, 8 in 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2715480966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: pgmap v268: 324 pgs: 14 creating+peering, 74 unknown, 236 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2742625329' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: osdmap e208: 8 total, 8 up, 8 in 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]: dispatch 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi045-74470-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-25", "mode": "writeback"}]': finished 2024-04-14T15:15:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: osdmap e209: 8 total, 8 up, 8 in 2024-04-14T15:15:00.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2715480966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.56678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:00.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:00.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: pgmap v271: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: osdmap e210: 8 total, 8 up, 8 in 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: pgmap v271: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: osdmap e210: 8 total, 8 up, 8 in 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.300 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:02 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: pgmap v271: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: from='client.56678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: osdmap e210: 8 total, 8 up, 8 in 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]: dispatch 2024-04-14T15:15:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:02 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: osdmap e211: 8 total, 8 up, 8 in 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/804401632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: osdmap e212: 8 total, 8 up, 8 in 2024-04-14T15:15:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: osdmap e211: 8 total, 8 up, 8 in 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/804401632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: osdmap e212: 8 total, 8 up, 8 in 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-25"}]': finished 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: osdmap e211: 8 total, 8 up, 8 in 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/804401632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.56684 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.56684 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi045-74490-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: osdmap e212: 8 total, 8 up, 8 in 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2517062016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:03 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]: dispatch 2024-04-14T15:15:03.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:04 smithi071 ceph-mon[33150]: pgmap v274: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-04-14T15:15:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:04 smithi071 ceph-mon[33150]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:04 smithi071 ceph-mon[33150]: osdmap e213: 8 total, 8 up, 8 in 2024-04-14T15:15:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[37795]: pgmap v274: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-04-14T15:15:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[37795]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[37795]: osdmap e213: 8 total, 8 up, 8 in 2024-04-14T15:15:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[29624]: pgmap v274: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 511 B/s wr, 0 op/s 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[29624]: from='client.56666 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi045-74470-35"}]': finished 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[29624]: osdmap e213: 8 total, 8 up, 8 in 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: osdmap e214: 8 total, 8 up, 8 in 2024-04-14T15:15:05.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: osdmap e214: 8 total, 8 up, 8 in 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi045-74470-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: osdmap e214: 8 total, 8 up, 8 in 2024-04-14T15:15:05.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:06 smithi071 ceph-mon[33150]: pgmap v277: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:06 smithi071 ceph-mon[33150]: osdmap e215: 8 total, 8 up, 8 in 2024-04-14T15:15:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:06 smithi045 ceph-mon[37795]: pgmap v277: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:06.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:06 smithi045 ceph-mon[37795]: osdmap e215: 8 total, 8 up, 8 in 2024-04-14T15:15:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:06 smithi045 ceph-mon[29624]: pgmap v277: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:06 smithi045 ceph-mon[29624]: osdmap e215: 8 total, 8 up, 8 in 2024-04-14T15:15:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: osdmap e216: 8 total, 8 up, 8 in 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: osdmap e216: 8 total, 8 up, 8 in 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi045-74470-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: osdmap e216: 8 total, 8 up, 8 in 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: pgmap v280: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: osdmap e217: 8 total, 8 up, 8 in 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:08.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: pgmap v280: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: osdmap e217: 8 total, 8 up, 8 in 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: pgmap v280: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:15:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: osdmap e217: 8 total, 8 up, 8 in 2024-04-14T15:15:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:08.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]: dispatch 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]': finished 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: osdmap e218: 8 total, 8 up, 8 in 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1095115399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:09 smithi071 ceph-mon[33150]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]': finished 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: osdmap e218: 8 total, 8 up, 8 in 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1095115399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[29624]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-27", "mode": "writeback"}]': finished 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: osdmap e218: 8 total, 8 up, 8 in 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1095115399' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:09 smithi045 ceph-mon[37795]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:10 smithi071 ceph-mon[33150]: pgmap v283: 236 pgs: 20 unknown, 216 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:10 smithi071 ceph-mon[33150]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:10 smithi071 ceph-mon[33150]: osdmap e219: 8 total, 8 up, 8 in 2024-04-14T15:15:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[29624]: pgmap v283: 236 pgs: 20 unknown, 216 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[29624]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[29624]: osdmap e219: 8 total, 8 up, 8 in 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[37795]: pgmap v283: 236 pgs: 20 unknown, 216 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[37795]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi045-74490-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[37795]: osdmap e219: 8 total, 8 up, 8 in 2024-04-14T15:15:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]: dispatch 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: osdmap e220: 8 total, 8 up, 8 in 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:11.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:11 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:11.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: osdmap e220: 8 total, 8 up, 8 in 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3536793070' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi045-74470-36"}]': finished 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: osdmap e220: 8 total, 8 up, 8 in 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:11.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:11.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:11 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: pgmap v286: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: osdmap e221: 8 total, 8 up, 8 in 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3233031862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:12.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:12 smithi071 ceph-mon[33150]: from='client.48106 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: pgmap v286: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-04-14T15:15:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: osdmap e221: 8 total, 8 up, 8 in 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3233031862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[29624]: from='client.48106 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: pgmap v286: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-27"}]': finished 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: osdmap e221: 8 total, 8 up, 8 in 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:12.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3233031862' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:12.500 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:12 smithi045 ceph-mon[37795]: from='client.48106 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:13 smithi071 ceph-mon[33150]: from='client.48106 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:13 smithi071 ceph-mon[33150]: osdmap e222: 8 total, 8 up, 8 in 2024-04-14T15:15:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[37795]: from='client.48106 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[37795]: osdmap e222: 8 total, 8 up, 8 in 2024-04-14T15:15:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[29624]: from='client.48106 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi045-74490-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:13 smithi045 ceph-mon[29624]: osdmap e222: 8 total, 8 up, 8 in 2024-04-14T15:15:14.083 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: pgmap v289: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: osdmap e223: 8 total, 8 up, 8 in 2024-04-14T15:15:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:14 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: pgmap v289: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: osdmap e223: 8 total, 8 up, 8 in 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: pgmap v289: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi045-74470-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: osdmap e223: 8 total, 8 up, 8 in 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:14.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:14 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: osdmap e224: 8 total, 8 up, 8 in 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:15 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: osdmap e224: 8 total, 8 up, 8 in 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: osdmap e224: 8 total, 8 up, 8 in 2024-04-14T15:15:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:15.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:15 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: pgmap v292: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: osdmap e225: 8 total, 8 up, 8 in 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:16 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: pgmap v292: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: osdmap e225: 8 total, 8 up, 8 in 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: pgmap v292: 236 pgs: 40 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:15:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: osdmap e225: 8 total, 8 up, 8 in 2024-04-14T15:15:16.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: pgmap v295: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: osdmap e226: 8 total, 8 up, 8 in 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:18.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: pgmap v295: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: osdmap e226: 8 total, 8 up, 8 in 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: pgmap v295: 228 pgs: 32 unknown, 196 active+clean; 584 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: osdmap e226: 8 total, 8 up, 8 in 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3230402913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]: dispatch 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:18.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: osdmap e227: 8 total, 8 up, 8 in 2024-04-14T15:15:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:19.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:19 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:19.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: osdmap e227: 8 total, 8 up, 8 in 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.48109 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi045-74470-37"}]': finished 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: osdmap e227: 8 total, 8 up, 8 in 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:19.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:19 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: pgmap v298: 236 pgs: 11 unknown, 225 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: osdmap e228: 8 total, 8 up, 8 in 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]': finished 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: osdmap e229: 8 total, 8 up, 8 in 2024-04-14T15:15:20.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:20 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: pgmap v298: 236 pgs: 11 unknown, 225 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: osdmap e228: 8 total, 8 up, 8 in 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]': finished 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: osdmap e229: 8 total, 8 up, 8 in 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: pgmap v298: 236 pgs: 11 unknown, 225 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi045-74470-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: osdmap e228: 8 total, 8 up, 8 in 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:20.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]: dispatch 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-29", "mode": "writeback"}]': finished 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/534295333' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: osdmap e229: 8 total, 8 up, 8 in 2024-04-14T15:15:20.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:20 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]: dispatch 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: pgmap v301: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: osdmap e230: 8 total, 8 up, 8 in 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:22.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:22 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: pgmap v301: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: osdmap e230: 8 total, 8 up, 8 in 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: pgmap v301: 228 pgs: 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi045-74470-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-46"}]': finished 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: osdmap e230: 8 total, 8 up, 8 in 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:22.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:22 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: osdmap e231: 8 total, 8 up, 8 in 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: osdmap e231: 8 total, 8 up, 8 in 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi045-74490-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: osdmap e231: 8 total, 8 up, 8 in 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:23.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:24.033 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: pgmap v304: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: osdmap e232: 8 total, 8 up, 8 in 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: osdmap e233: 8 total, 8 up, 8 in 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:24 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: pgmap v304: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: osdmap e232: 8 total, 8 up, 8 in 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: osdmap e233: 8 total, 8 up, 8 in 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: pgmap v304: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 595 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: osdmap e232: 8 total, 8 up, 8 in 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi045-74490-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-29"}]': finished 2024-04-14T15:15:24.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:24.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: osdmap e233: 8 total, 8 up, 8 in 2024-04-14T15:15:24.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3696607539' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:24.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:24 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]: dispatch 2024-04-14T15:15:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:25 smithi071 ceph-mon[33150]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:25 smithi071 ceph-mon[33150]: osdmap e234: 8 total, 8 up, 8 in 2024-04-14T15:15:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[37795]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[37795]: osdmap e234: 8 total, 8 up, 8 in 2024-04-14T15:15:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[29624]: from='client.56720 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi045-74470-38"}]': finished 2024-04-14T15:15:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:25 smithi045 ceph-mon[29624]: osdmap e234: 8 total, 8 up, 8 in 2024-04-14T15:15:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: pgmap v307: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-04-14T15:15:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: osdmap e235: 8 total, 8 up, 8 in 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.359 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:26 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: pgmap v307: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-04-14T15:15:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: osdmap e235: 8 total, 8 up, 8 in 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: pgmap v307: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi045-74470-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: osdmap e235: 8 total, 8 up, 8 in 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:26.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:26.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: osdmap e236: 8 total, 8 up, 8 in 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: osdmap e236: 8 total, 8 up, 8 in 2024-04-14T15:15:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: pgmap v310: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2155511599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: osdmap e236: 8 total, 8 up, 8 in 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]: dispatch 2024-04-14T15:15:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:28.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: osdmap e237: 8 total, 8 up, 8 in 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:29.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:29 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: osdmap e237: 8 total, 8 up, 8 in 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi045-74470-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.48127 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi045-74490-47"}]': finished 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: osdmap e237: 8 total, 8 up, 8 in 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:29.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:29 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: pgmap v313: 236 pgs: 5 creating+peering, 14 unknown, 217 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: osdmap e238: 8 total, 8 up, 8 in 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]': finished 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: osdmap e239: 8 total, 8 up, 8 in 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:30.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:30 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: pgmap v313: 236 pgs: 5 creating+peering, 14 unknown, 217 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: osdmap e238: 8 total, 8 up, 8 in 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]': finished 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: osdmap e239: 8 total, 8 up, 8 in 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: pgmap v313: 236 pgs: 5 creating+peering, 14 unknown, 217 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi045-74490-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: osdmap e238: 8 total, 8 up, 8 in 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]: dispatch 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-31", "mode": "writeback"}]': finished 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: osdmap e239: 8 total, 8 up, 8 in 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:30 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:31.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: osdmap e240: 8 total, 8 up, 8 in 2024-04-14T15:15:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: osdmap e240: 8 total, 8 up, 8 in 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi045-74490-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: osdmap e240: 8 total, 8 up, 8 in 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/98428435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]: dispatch 2024-04-14T15:15:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]: dispatch 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: pgmap v316: 228 pgs: 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: osdmap e241: 8 total, 8 up, 8 in 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:32 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: pgmap v316: 228 pgs: 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: osdmap e241: 8 total, 8 up, 8 in 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:32.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: pgmap v316: 228 pgs: 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: from='client.56732 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi045-74470-39"}]': finished 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-31"}]': finished 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: osdmap e241: 8 total, 8 up, 8 in 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:32.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:32 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:33.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: osdmap e242: 8 total, 8 up, 8 in 2024-04-14T15:15:33.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:33.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:33.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: osdmap e242: 8 total, 8 up, 8 in 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi045-74470-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: osdmap e242: 8 total, 8 up, 8 in 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:33 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.053 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: pgmap v319: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: osdmap e243: 8 total, 8 up, 8 in 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: pgmap v319: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: osdmap e243: 8 total, 8 up, 8 in 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: pgmap v319: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3806140657' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: osdmap e243: 8 total, 8 up, 8 in 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]: dispatch 2024-04-14T15:15:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:34.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: osdmap e244: 8 total, 8 up, 8 in 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:35 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: osdmap e244: 8 total, 8 up, 8 in 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi045-74470-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi045-74490-48"}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: osdmap e244: 8 total, 8 up, 8 in 2024-04-14T15:15:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:35.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:35 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: pgmap v322: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: osdmap e245: 8 total, 8 up, 8 in 2024-04-14T15:15:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: pgmap v322: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: osdmap e245: 8 total, 8 up, 8 in 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: pgmap v322: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi045-74490-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: osdmap e245: 8 total, 8 up, 8 in 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: pgmap v325: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: osdmap e246: 8 total, 8 up, 8 in 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:38 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: pgmap v325: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: osdmap e246: 8 total, 8 up, 8 in 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: pgmap v325: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: osdmap e246: 8 total, 8 up, 8 in 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]: dispatch 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:38.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:38 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]': finished 2024-04-14T15:15:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: osdmap e247: 8 total, 8 up, 8 in 2024-04-14T15:15:39.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:39.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]': finished 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: osdmap e247: 8 total, 8 up, 8 in 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi045-74490-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-33", "mode": "writeback"}]': finished 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2478111741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: osdmap e247: 8 total, 8 up, 8 in 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]: dispatch 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:39.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: pgmap v328: 236 pgs: 5 creating+peering, 15 unknown, 216 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: osdmap e248: 8 total, 8 up, 8 in 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:40 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: pgmap v328: 236 pgs: 5 creating+peering, 15 unknown, 216 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: osdmap e248: 8 total, 8 up, 8 in 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: pgmap v328: 236 pgs: 5 creating+peering, 15 unknown, 216 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.48145 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi045-74470-40"}]': finished 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: osdmap e248: 8 total, 8 up, 8 in 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:40.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:40 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: osdmap e249: 8 total, 8 up, 8 in 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: osdmap e249: 8 total, 8 up, 8 in 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-33"}]': finished 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi045-74470-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: osdmap e249: 8 total, 8 up, 8 in 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: osdmap e250: 8 total, 8 up, 8 in 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: osdmap e251: 8 total, 8 up, 8 in 2024-04-14T15:15:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:42.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:15:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: osdmap e250: 8 total, 8 up, 8 in 2024-04-14T15:15:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: osdmap e251: 8 total, 8 up, 8 in 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: pgmap v331: 228 pgs: 228 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: osdmap e250: 8 total, 8 up, 8 in 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]: dispatch 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi045-74470-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3073363647' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi045-74490-49"}]': finished 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: osdmap e251: 8 total, 8 up, 8 in 2024-04-14T15:15:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:42.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: osdmap e252: 8 total, 8 up, 8 in 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:43.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: osdmap e252: 8 total, 8 up, 8 in 2024-04-14T15:15:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:43.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi045-74490-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: osdmap e252: 8 total, 8 up, 8 in 2024-04-14T15:15:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:43.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:44.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: pgmap v334: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: osdmap e253: 8 total, 8 up, 8 in 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:44 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: pgmap v334: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: osdmap e253: 8 total, 8 up, 8 in 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: pgmap v334: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:44.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: osdmap e253: 8 total, 8 up, 8 in 2024-04-14T15:15:44.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:44.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:44.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:44 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: osdmap e254: 8 total, 8 up, 8 in 2024-04-14T15:15:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:45.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: osdmap e254: 8 total, 8 up, 8 in 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi045-74490-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: osdmap e254: 8 total, 8 up, 8 in 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3507645635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]: dispatch 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:45.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]: dispatch 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: pgmap v337: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]': finished 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: osdmap e255: 8 total, 8 up, 8 in 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:46 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: pgmap v337: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]': finished 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: osdmap e255: 8 total, 8 up, 8 in 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: pgmap v337: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.48157 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi045-74470-41"}]': finished 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-35", "mode": "writeback"}]': finished 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: osdmap e255: 8 total, 8 up, 8 in 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:46.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: pgmap v340: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: osdmap e256: 8 total, 8 up, 8 in 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:48 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: pgmap v340: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: osdmap e256: 8 total, 8 up, 8 in 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: pgmap v340: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi045-74470-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: osdmap e256: 8 total, 8 up, 8 in 2024-04-14T15:15:48.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:48.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:48.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:48 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: osdmap e257: 8 total, 8 up, 8 in 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:49.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:49 smithi071 ceph-mon[33150]: osdmap e258: 8 total, 8 up, 8 in 2024-04-14T15:15:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: osdmap e257: 8 total, 8 up, 8 in 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[29624]: osdmap e258: 8 total, 8 up, 8 in 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1206060205' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: osdmap e257: 8 total, 8 up, 8 in 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]: dispatch 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi045-74470-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.56762 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi045-74490-50"}]': finished 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:49.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:49 smithi045 ceph-mon[37795]: osdmap e258: 8 total, 8 up, 8 in 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: osdmap e259: 8 total, 8 up, 8 in 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3205360486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:50 smithi071 ceph-mon[33150]: from='client.56774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: osdmap e259: 8 total, 8 up, 8 in 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3205360486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[29624]: from='client.56774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: pgmap v343: 228 pgs: 228 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:50.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: osdmap e259: 8 total, 8 up, 8 in 2024-04-14T15:15:50.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3205360486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:50.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:50 smithi045 ceph-mon[37795]: from='client.56774 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: pgmap v346: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: from='client.56774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: osdmap e260: 8 total, 8 up, 8 in 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:52.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: pgmap v346: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: from='client.56774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: osdmap e260: 8 total, 8 up, 8 in 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: pgmap v346: 268 pgs: 40 unknown, 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: from='client.56774 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi045-74490-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: osdmap e260: 8 total, 8 up, 8 in 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:52.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:52.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7046 ms) 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6927 ms) 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7014 ms) 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6986 ms) 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7047 ms) 2024-04-14T15:15:53.075 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7039 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7026 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7016 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6885 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7012 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7029 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6969 ms) 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-04-14T15:15:53.076 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6991 ms) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7021 ms) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7014 ms) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7014 ms) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112036 ms total) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [----------] Global test environment tear-down 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (189660 ms total) 2024-04-14T15:15:53.077 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio: [ PASSED ] 42 tests. 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: osdmap e261: 8 total, 8 up, 8 in 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: osdmap e262: 8 total, 8 up, 8 in 2024-04-14T15:15:53.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: osdmap e261: 8 total, 8 up, 8 in 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: osdmap e262: 8 total, 8 up, 8 in 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: osdmap e261: 8 total, 8 up, 8 in 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1424921567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-35"}]': finished 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.56768 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi045-74470-42"}]': finished 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: osdmap e262: 8 total, 8 up, 8 in 2024-04-14T15:15:53.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:54.083 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:15:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:15:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:54 smithi071 ceph-mon[33150]: pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:54 smithi045 ceph-mon[37795]: pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:54 smithi045 ceph-mon[29624]: pgmap v349: 228 pgs: 228 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: osdmap e263: 8 total, 8 up, 8 in 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: osdmap e264: 8 total, 8 up, 8 in 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:55.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:55.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: osdmap e263: 8 total, 8 up, 8 in 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: osdmap e264: 8 total, 8 up, 8 in 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: osdmap e263: 8 total, 8 up, 8 in 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi045-74490-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: osdmap e264: 8 total, 8 up, 8 in 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:15:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:56 smithi071 ceph-mon[33150]: pgmap v352: 196 pgs: 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:56 smithi071 ceph-mon[33150]: osdmap e265: 8 total, 8 up, 8 in 2024-04-14T15:15:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[29624]: pgmap v352: 196 pgs: 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[29624]: osdmap e265: 8 total, 8 up, 8 in 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[37795]: pgmap v352: 196 pgs: 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[37795]: osdmap e265: 8 total, 8 up, 8 in 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:15:56.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:15:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:15:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:15:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: pgmap v355: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: osdmap e266: 8 total, 8 up, 8 in 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:58.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: pgmap v355: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: osdmap e266: 8 total, 8 up, 8 in 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: pgmap v355: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: osdmap e266: 8 total, 8 up, 8 in 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:15:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:15:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: osdmap e267: 8 total, 8 up, 8 in 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:15:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]': finished 2024-04-14T15:15:59.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: osdmap e267: 8 total, 8 up, 8 in 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]': finished 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: osdmap e267: 8 total, 8 up, 8 in 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]: dispatch 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]: dispatch 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1327970486' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi045-74490-52"}]': finished 2024-04-14T15:15:59.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:15:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-37", "mode": "writeback"}]': finished 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: pgmap v358: 228 pgs: 9 unknown, 219 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: osdmap e268: 8 total, 8 up, 8 in 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: osdmap e269: 8 total, 8 up, 8 in 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: pgmap v358: 228 pgs: 9 unknown, 219 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: osdmap e268: 8 total, 8 up, 8 in 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: osdmap e269: 8 total, 8 up, 8 in 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: pgmap v358: 228 pgs: 9 unknown, 219 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: osdmap e268: 8 total, 8 up, 8 in 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: osdmap e269: 8 total, 8 up, 8 in 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:00.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]: dispatch 2024-04-14T15:16:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:01 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:16:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:01 smithi071 ceph-mon[33150]: osdmap e270: 8 total, 8 up, 8 in 2024-04-14T15:16:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:16:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[37795]: osdmap e270: 8 total, 8 up, 8 in 2024-04-14T15:16:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-37"}]': finished 2024-04-14T15:16:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:01 smithi045 ceph-mon[29624]: osdmap e270: 8 total, 8 up, 8 in 2024-04-14T15:16:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:02 smithi071 ceph-mon[33150]: pgmap v361: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:02 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:02 smithi071 ceph-mon[33150]: osdmap e271: 8 total, 8 up, 8 in 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[29624]: pgmap v361: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[29624]: osdmap e271: 8 total, 8 up, 8 in 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[37795]: pgmap v361: 228 pgs: 228 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi045-74490-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:02.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:02 smithi045 ceph-mon[37795]: osdmap e271: 8 total, 8 up, 8 in 2024-04-14T15:16:04.040 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:04 smithi071 ceph-mon[33150]: pgmap v364: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:04 smithi071 ceph-mon[33150]: osdmap e272: 8 total, 8 up, 8 in 2024-04-14T15:16:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:04 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[29624]: pgmap v364: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[29624]: osdmap e272: 8 total, 8 up, 8 in 2024-04-14T15:16:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[37795]: pgmap v364: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[37795]: osdmap e272: 8 total, 8 up, 8 in 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:04 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: osdmap e273: 8 total, 8 up, 8 in 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:05 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: osdmap e273: 8 total, 8 up, 8 in 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: osdmap e273: 8 total, 8 up, 8 in 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:05.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:05 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: pgmap v367: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: osdmap e274: 8 total, 8 up, 8 in 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: pgmap v367: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: osdmap e274: 8 total, 8 up, 8 in 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: pgmap v367: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: osdmap e274: 8 total, 8 up, 8 in 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/36185296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]: dispatch 2024-04-14T15:16:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: osdmap e275: 8 total, 8 up, 8 in 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]': finished 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:07.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:07 smithi071 ceph-mon[33150]: osdmap e276: 8 total, 8 up, 8 in 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: osdmap e275: 8 total, 8 up, 8 in 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]': finished 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[37795]: osdmap e276: 8 total, 8 up, 8 in 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi045-74490-53"}]': finished 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: osdmap e275: 8 total, 8 up, 8 in 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]: dispatch 2024-04-14T15:16:07.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-39", "mode": "writeback"}]': finished 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi045-74490-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:07.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:07 smithi045 ceph-mon[29624]: osdmap e276: 8 total, 8 up, 8 in 2024-04-14T15:16:08.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: pgmap v370: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:08.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.409 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: pgmap v370: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: pgmap v370: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:16:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: osdmap e277: 8 total, 8 up, 8 in 2024-04-14T15:16:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:09.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:09.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:09 smithi071 ceph-mon[33150]: osdmap e278: 8 total, 8 up, 8 in 2024-04-14T15:16:09.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: osdmap e277: 8 total, 8 up, 8 in 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[29624]: osdmap e278: 8 total, 8 up, 8 in 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: osdmap e277: 8 total, 8 up, 8 in 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]: dispatch 2024-04-14T15:16:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi045-74490-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-39"}]': finished 2024-04-14T15:16:09.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:09 smithi045 ceph-mon[37795]: osdmap e278: 8 total, 8 up, 8 in 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: seed 74757 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (419 ms) 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (10344 ms) 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (66 ms) 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (10829 ms total) 2024-04-14T15:16:10.064 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7047 ms) 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7962 ms) 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10046 ms) 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: my_snaps [3] 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: my_snaps [4,3] 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: my_snaps [5,4,3] 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-04-14T15:16:10.065 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: promoting some heads 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: promoting from clones for snap 6 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: promoting from clones for snap 5 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: promoting from clones for snap 4 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: promoting from clones for snap 3 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: waiting for scrubs... 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: done waiting 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46037 ms) 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9923 ms) 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7986 ms) 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8053 ms) 2024-04-14T15:16:10.066 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8050 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10019 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8891 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10031 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (11976 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7999 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8031 ms) 2024-04-14T15:16:10.067 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-04-14T15:16:10.068 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13028 ms) 2024-04-14T15:16:10.068 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-04-14T15:16:10.068 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7947 ms) 2024-04-14T15:16:10.068 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-04-14T15:16:10.068 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8035 ms) 2024-04-14T15:16:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:10 smithi071 ceph-mon[33150]: pgmap v373: 228 pgs: 11 creating+peering, 217 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:10 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:10.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:10 smithi071 ceph-mon[33150]: osdmap e279: 8 total, 8 up, 8 in 2024-04-14T15:16:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[29624]: pgmap v373: 228 pgs: 11 creating+peering, 217 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[29624]: osdmap e279: 8 total, 8 up, 8 in 2024-04-14T15:16:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[37795]: pgmap v373: 228 pgs: 11 creating+peering, 217 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:10 smithi045 ceph-mon[37795]: osdmap e279: 8 total, 8 up, 8 in 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: pgmap v376: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: osdmap e280: 8 total, 8 up, 8 in 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: pgmap v376: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: osdmap e280: 8 total, 8 up, 8 in 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: pgmap v376: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: osdmap e280: 8 total, 8 up, 8 in 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:12.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: osdmap e281: 8 total, 8 up, 8 in 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: osdmap e281: 8 total, 8 up, 8 in 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: osdmap e281: 8 total, 8 up, 8 in 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2008303411' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]: dispatch 2024-04-14T15:16:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:13.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:14.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: pgmap v379: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: osdmap e282: 8 total, 8 up, 8 in 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: osdmap e283: 8 total, 8 up, 8 in 2024-04-14T15:16:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: pgmap v379: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: osdmap e282: 8 total, 8 up, 8 in 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: osdmap e283: 8 total, 8 up, 8 in 2024-04-14T15:16:14.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: pgmap v379: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.56792 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi045-74490-54"}]': finished 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: osdmap e282: 8 total, 8 up, 8 in 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:14.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:14.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi045-74490-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:14.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: osdmap e283: 8 total, 8 up, 8 in 2024-04-14T15:16:14.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:15 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:15.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: pgmap v382: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: osdmap e284: 8 total, 8 up, 8 in 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:16:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:16 smithi071 ceph-mon[33150]: osdmap e285: 8 total, 8 up, 8 in 2024-04-14T15:16:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: pgmap v382: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: osdmap e284: 8 total, 8 up, 8 in 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[29624]: osdmap e285: 8 total, 8 up, 8 in 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: pgmap v382: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: osdmap e284: 8 total, 8 up, 8 in 2024-04-14T15:16:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:16:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:16:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi045-74490-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:16:16.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-mon[37795]: osdmap e285: 8 total, 8 up, 8 in 2024-04-14T15:16:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: pgmap v385: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: osdmap e286: 8 total, 8 up, 8 in 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:16:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: pgmap v385: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: osdmap e286: 8 total, 8 up, 8 in 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:16:17.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:16:17.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: pgmap v385: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: osdmap e286: 8 total, 8 up, 8 in 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]: dispatch 2024-04-14T15:16:17.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:16:17.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:16:17.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:16:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]': finished 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[37795]: osdmap e287: 8 total, 8 up, 8 in 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]': finished 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[29624]: osdmap e287: 8 total, 8 up, 8 in 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:18 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:18 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-41"}]': finished 2024-04-14T15:16:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:18 smithi071 ceph-mon[33150]: osdmap e287: 8 total, 8 up, 8 in 2024-04-14T15:16:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:18 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:20 smithi071 ceph-mon[33150]: pgmap v388: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:20 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:20 smithi071 ceph-mon[33150]: osdmap e288: 8 total, 8 up, 8 in 2024-04-14T15:16:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:20 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[37795]: pgmap v388: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[37795]: osdmap e288: 8 total, 8 up, 8 in 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[29624]: pgmap v388: 228 pgs: 228 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[29624]: osdmap e288: 8 total, 8 up, 8 in 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3665883023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:20 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]: dispatch 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: osdmap e289: 8 total, 8 up, 8 in 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: osdmap e290: 8 total, 8 up, 8 in 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-6","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:21.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:21.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: osdmap e289: 8 total, 8 up, 8 in 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: osdmap e290: 8 total, 8 up, 8 in 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-6","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.56798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi045-74490-55"}]': finished 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: osdmap e289: 8 total, 8 up, 8 in 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi045-74490-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: osdmap e290: 8 total, 8 up, 8 in 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:21.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-6","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:21.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:21.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:22 smithi071 ceph-mon[33150]: pgmap v391: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:22 smithi071 ceph-mon[33150]: osdmap e291: 8 total, 8 up, 8 in 2024-04-14T15:16:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[37795]: pgmap v391: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[37795]: osdmap e291: 8 total, 8 up, 8 in 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[29624]: pgmap v391: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[29624]: osdmap e291: 8 total, 8 up, 8 in 2024-04-14T15:16:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]: dispatch 2024-04-14T15:16:23.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: pgmap v394: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]': finished 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: osdmap e292: 8 total, 8 up, 8 in 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:24 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: pgmap v394: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]': finished 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: osdmap e292: 8 total, 8 up, 8 in 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: pgmap v394: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi045-74490-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_count","val": "8"}]': finished 2024-04-14T15:16:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: osdmap e292: 8 total, 8 up, 8 in 2024-04-14T15:16:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:24.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:24 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:25.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-04-14T15:16:25.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: hmm, no HitSet yet 2024-04-14T15:16:25.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-04-14T15:16:25.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9015 ms) 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg_num = 32 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 0 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 1 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 2 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 3 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 4 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 5 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 6 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 7 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 8 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 9 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 10 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 11 ls 1713107786,0 2024-04-14T15:16:25.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 12 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 13 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 14 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 15 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 16 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 17 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 18 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 19 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 20 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 21 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 22 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 23 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 24 ls 1713107786,0 2024-04-14T15:16:25.326 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 25 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 26 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 27 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 28 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 29 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 30 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg 31 ls 1713107786,0 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: pg_num = 32 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-04-14T15:16:25.327 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-04-14T15:16:25.328 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-04-14T15:16:25.329 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-04-14T15:16:25.330 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-04-14T15:16:25.331 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-04-14T15:16:25.332 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-04-14T15:16:25.333 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-04-14T15:16:25.334 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-04-14T15:16:25.335 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-04-14T15:16:25.336 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-04-14T15:16:25.337 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-04-14T15:16:25.338 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-04-14T15:16:25.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: osdmap e293: 8 total, 8 up, 8 in 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: osdmap e294: 8 total, 8 up, 8 in 2024-04-14T15:16:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: osdmap e293: 8 total, 8 up, 8 in 2024-04-14T15:16:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: osdmap e294: 8 total, 8 up, 8 in 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: osdmap e293: 8 total, 8 up, 8 in 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: osdmap e294: 8 total, 8 up, 8 in 2024-04-14T15:16:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: pgmap v397: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-43","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]': finished 2024-04-14T15:16:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: osdmap e295: 8 total, 8 up, 8 in 2024-04-14T15:16:26.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: pgmap v397: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-43","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]': finished 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: osdmap e295: 8 total, 8 up, 8 in 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: pgmap v397: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi045-74757-43","var": "pg_num","format": "json"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]: dispatch 2024-04-14T15:16:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-43"}]': finished 2024-04-14T15:16:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: osdmap e295: 8 total, 8 up, 8 in 2024-04-14T15:16:26.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]: dispatch 2024-04-14T15:16:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: osdmap e296: 8 total, 8 up, 8 in 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: osdmap e296: 8 total, 8 up, 8 in 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: pgmap v400: 228 pgs: 228 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3181473547' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi045-74490-56"}]': finished 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: osdmap e296: 8 total, 8 up, 8 in 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:28.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: osdmap e297: 8 total, 8 up, 8 in 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:29 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: osdmap e297: 8 total, 8 up, 8 in 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi045-74490-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: osdmap e297: 8 total, 8 up, 8 in 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:29 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: pgmap v403: 228 pgs: 16 creating+peering, 16 unknown, 196 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: osdmap e298: 8 total, 8 up, 8 in 2024-04-14T15:16:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: pgmap v403: 228 pgs: 16 creating+peering, 16 unknown, 196 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: osdmap e298: 8 total, 8 up, 8 in 2024-04-14T15:16:30.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: pgmap v403: 228 pgs: 16 creating+peering, 16 unknown, 196 active+clean; 8.4 MiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:30.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:30.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: osdmap e298: 8 total, 8 up, 8 in 2024-04-14T15:16:30.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[37795]: osdmap e299: 8 total, 8 up, 8 in 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[29624]: osdmap e299: 8 total, 8 up, 8 in 2024-04-14T15:16:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:31.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:31 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi045-74490-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:31 smithi071 ceph-mon[33150]: osdmap e299: 8 total, 8 up, 8 in 2024-04-14T15:16:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:16:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[37795]: pgmap v406: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:16:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[37795]: osdmap e300: 8 total, 8 up, 8 in 2024-04-14T15:16:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:32.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[29624]: pgmap v406: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[29624]: osdmap e300: 8 total, 8 up, 8 in 2024-04-14T15:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:32.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:32 smithi071 ceph-mon[33150]: pgmap v406: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:16:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:32 smithi071 ceph-mon[33150]: osdmap e300: 8 total, 8 up, 8 in 2024-04-14T15:16:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: osdmap e301: 8 total, 8 up, 8 in 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:33.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:16:33.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: osdmap e301: 8 total, 8 up, 8 in 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: osdmap e301: 8 total, 8 up, 8 in 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:33.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:33.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: osdmap e302: 8 total, 8 up, 8 in 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[37795]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: osdmap e302: 8 total, 8 up, 8 in 2024-04-14T15:16:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:34 smithi045 ceph-mon[29624]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: pgmap v409: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3261138496' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: osdmap e302: 8 total, 8 up, 8 in 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]: dispatch 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:16:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:34 smithi071 ceph-mon[33150]: from='client.48211 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi045-74490-57"}]': finished 2024-04-14T15:16:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: osdmap e303: 8 total, 8 up, 8 in 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: osdmap e303: 8 total, 8 up, 8 in 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:35 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: osdmap e303: 8 total, 8 up, 8 in 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:35.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:35.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:35 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[37795]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:36.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[37795]: osdmap e304: 8 total, 8 up, 8 in 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[29624]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[29624]: osdmap e304: 8 total, 8 up, 8 in 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:36 smithi071 ceph-mon[33150]: pgmap v412: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:36.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:36 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi045-74490-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:36.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:36 smithi071 ceph-mon[33150]: osdmap e304: 8 total, 8 up, 8 in 2024-04-14T15:16:36.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:36 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:37.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[37795]: osdmap e305: 8 total, 8 up, 8 in 2024-04-14T15:16:37.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:37.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[37795]: osdmap e306: 8 total, 8 up, 8 in 2024-04-14T15:16:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[29624]: osdmap e305: 8 total, 8 up, 8 in 2024-04-14T15:16:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:37.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:37 smithi045 ceph-mon[29624]: osdmap e306: 8 total, 8 up, 8 in 2024-04-14T15:16:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:37 smithi071 ceph-mon[33150]: osdmap e305: 8 total, 8 up, 8 in 2024-04-14T15:16:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:37 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi045-74490-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:37 smithi071 ceph-mon[33150]: osdmap e306: 8 total, 8 up, 8 in 2024-04-14T15:16:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:38 smithi045 ceph-mon[29624]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:38 smithi045 ceph-mon[29624]: osdmap e307: 8 total, 8 up, 8 in 2024-04-14T15:16:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:38 smithi045 ceph-mon[37795]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:38 smithi045 ceph-mon[37795]: osdmap e307: 8 total, 8 up, 8 in 2024-04-14T15:16:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:38 smithi071 ceph-mon[33150]: pgmap v415: 228 pgs: 228 active+clean; 8.4 MiB data, 596 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:38 smithi071 ceph-mon[33150]: osdmap e307: 8 total, 8 up, 8 in 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[29624]: pgmap v418: 236 pgs: 6 unknown, 230 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[29624]: osdmap e308: 8 total, 8 up, 8 in 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[37795]: pgmap v418: 236 pgs: 6 unknown, 230 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[37795]: osdmap e308: 8 total, 8 up, 8 in 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:40 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:40 smithi071 ceph-mon[33150]: pgmap v418: 236 pgs: 6 unknown, 230 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:40 smithi071 ceph-mon[33150]: osdmap e308: 8 total, 8 up, 8 in 2024-04-14T15:16:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:40 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:40 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[37795]: osdmap e309: 8 total, 8 up, 8 in 2024-04-14T15:16:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:41.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[29624]: osdmap e309: 8 total, 8 up, 8 in 2024-04-14T15:16:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:41.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:41 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:41 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:41 smithi071 ceph-mon[33150]: osdmap e309: 8 total, 8 up, 8 in 2024-04-14T15:16:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2264860851' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:41 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]: dispatch 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: osdmap e310: 8 total, 8 up, 8 in 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:42.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:16:42.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:42.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: osdmap e310: 8 total, 8 up, 8 in 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:42.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:42 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: pgmap v421: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.56816 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi045-74490-58"}]': finished 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: osdmap e310: 8 total, 8 up, 8 in 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:42.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:42 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[37795]: osdmap e311: 8 total, 8 up, 8 in 2024-04-14T15:16:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[29624]: osdmap e311: 8 total, 8 up, 8 in 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-mon[33150]: osdmap e311: 8 total, 8 up, 8 in 2024-04-14T15:16:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:44.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[37795]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-14T15:16:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[37795]: osdmap e312: 8 total, 8 up, 8 in 2024-04-14T15:16:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]: dispatch 2024-04-14T15:16:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[29624]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-14T15:16:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[29624]: osdmap e312: 8 total, 8 up, 8 in 2024-04-14T15:16:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]: dispatch 2024-04-14T15:16:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:44 smithi071 ceph-mon[33150]: pgmap v424: 228 pgs: 228 active+clean; 8.4 MiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-14T15:16:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:44 smithi071 ceph-mon[33150]: osdmap e312: 8 total, 8 up, 8 in 2024-04-14T15:16:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]: dispatch 2024-04-14T15:16:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[29624]: osdmap e313: 8 total, 8 up, 8 in 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[37795]: osdmap e313: 8 total, 8 up, 8 in 2024-04-14T15:16:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:45 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:45 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi045-74490-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:45 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "109.12", "id": [3, 0]}]': finished 2024-04-14T15:16:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:45 smithi071 ceph-mon[33150]: osdmap e313: 8 total, 8 up, 8 in 2024-04-14T15:16:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:45 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:46 smithi045 ceph-mon[29624]: pgmap v427: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:46 smithi045 ceph-mon[29624]: osdmap e314: 8 total, 8 up, 8 in 2024-04-14T15:16:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:46 smithi045 ceph-mon[37795]: pgmap v427: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:46 smithi045 ceph-mon[37795]: osdmap e314: 8 total, 8 up, 8 in 2024-04-14T15:16:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:46 smithi071 ceph-mon[33150]: pgmap v427: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:46 smithi071 ceph-mon[33150]: osdmap e314: 8 total, 8 up, 8 in 2024-04-14T15:16:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: osdmap e315: 8 total, 8 up, 8 in 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: osdmap e315: 8 total, 8 up, 8 in 2024-04-14T15:16:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: osdmap e315: 8 total, 8 up, 8 in 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]: dispatch 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7462ddf6:::.RoundTripAppendPP (2980 ms) 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3005 ms) 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3029 ms) 2024-04-14T15:16:48.204 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3008 ms) 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: setting pool EIO 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: max_success 100, min_failed 101 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4036 ms) 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3008 ms) 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (117670 ms total) 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-04-14T15:16:48.205 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3023 ms) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9016 ms) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14903 ms) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3005 ms) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29947 ms total) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3010 ms) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3011 ms total) 2024-04-14T15:16:48.206 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13991 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6993 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7029 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7008 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3004 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7012 ms) 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-04-14T15:16:48.207 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6973 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7018 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7010 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7012 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7032 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7036 ms) 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-04-14T15:16:48.208 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7042 ms) 2024-04-14T15:16:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]': finished 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: osdmap e316: 8 total, 8 up, 8 in 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]': finished 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: osdmap e316: 8 total, 8 up, 8 in 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:48 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: pgmap v430: 228 pgs: 228 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-45"}]': finished 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: osdmap e316: 8 total, 8 up, 8 in 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2763954512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:48 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]: dispatch 2024-04-14T15:16:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:49 smithi045 ceph-mon[29624]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:49 smithi045 ceph-mon[29624]: osdmap e317: 8 total, 8 up, 8 in 2024-04-14T15:16:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:49 smithi045 ceph-mon[37795]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:49 smithi045 ceph-mon[37795]: osdmap e317: 8 total, 8 up, 8 in 2024-04-14T15:16:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:49 smithi071 ceph-mon[33150]: from='client.56822 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi045-74490-59"}]': finished 2024-04-14T15:16:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:49 smithi071 ceph-mon[33150]: osdmap e317: 8 total, 8 up, 8 in 2024-04-14T15:16:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: pgmap v433: 196 pgs: 196 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: osdmap e318: 8 total, 8 up, 8 in 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3164831820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[29624]: from='client.56828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: pgmap v433: 196 pgs: 196 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: osdmap e318: 8 total, 8 up, 8 in 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3164831820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:50 smithi045 ceph-mon[37795]: from='client.56828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: pgmap v433: 196 pgs: 196 active+clean; 8.4 MiB data, 606 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: osdmap e318: 8 total, 8 up, 8 in 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3164831820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:50 smithi071 ceph-mon[33150]: from='client.56828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:16:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[29624]: from='client.56828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[29624]: osdmap e319: 8 total, 8 up, 8 in 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[37795]: from='client.56828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[37795]: osdmap e319: 8 total, 8 up, 8 in 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:51 smithi071 ceph-mon[33150]: from='client.56828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi045-74490-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:16:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:51 smithi071 ceph-mon[33150]: osdmap e319: 8 total, 8 up, 8 in 2024-04-14T15:16:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:16:52.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: pgmap v436: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:52.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: osdmap e320: 8 total, 8 up, 8 in 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: pgmap v436: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: osdmap e320: 8 total, 8 up, 8 in 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:52.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:52 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: pgmap v436: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:16:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: osdmap e320: 8 total, 8 up, 8 in 2024-04-14T15:16:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:52.509 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:52 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:53.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: osdmap e321: 8 total, 8 up, 8 in 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]': finished 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: osdmap e321: 8 total, 8 up, 8 in 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:53.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]': finished 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi045-74490-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: osdmap e321: 8 total, 8 up, 8 in 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]: dispatch 2024-04-14T15:16:53.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:16:53.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-47", "mode": "writeback"}]': finished 2024-04-14T15:16:54.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:16:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:16:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: osdmap e322: 8 total, 8 up, 8 in 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: osdmap e323: 8 total, 8 up, 8 in 2024-04-14T15:16:54.504 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: osdmap e322: 8 total, 8 up, 8 in 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: osdmap e323: 8 total, 8 up, 8 in 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:54.505 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: pgmap v439: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 6 B/s, 2 objects/s recovering 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: osdmap e322: 8 total, 8 up, 8 in 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi045-74490-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: osdmap e323: 8 total, 8 up, 8 in 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:16:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:55 smithi071 ceph-mon[33150]: osdmap e324: 8 total, 8 up, 8 in 2024-04-14T15:16:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[29624]: osdmap e324: 8 total, 8 up, 8 in 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[37795]: osdmap e324: 8 total, 8 up, 8 in 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: osdmap e325: 8 total, 8 up, 8 in 2024-04-14T15:16:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:56 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: osdmap e325: 8 total, 8 up, 8 in 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: osdmap e325: 8 total, 8 up, 8 in 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:16:56.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:16:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:16:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:16:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: osdmap e326: 8 total, 8 up, 8 in 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: osdmap e326: 8 total, 8 up, 8 in 2024-04-14T15:16:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: pgmap v445: 228 pgs: 228 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:16:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:16:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: osdmap e326: 8 total, 8 up, 8 in 2024-04-14T15:16:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3521304079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]: dispatch 2024-04-14T15:16:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:16:58.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: osdmap e327: 8 total, 8 up, 8 in 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:59.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:16:59 smithi071 ceph-mon[33150]: osdmap e328: 8 total, 8 up, 8 in 2024-04-14T15:16:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: osdmap e327: 8 total, 8 up, 8 in 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[29624]: osdmap e328: 8 total, 8 up, 8 in 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.56834 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi045-74490-61"}]': finished 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: osdmap e327: 8 total, 8 up, 8 in 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:16:59.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:16:59.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:16:59.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:16:59 smithi045 ceph-mon[37795]: osdmap e328: 8 total, 8 up, 8 in 2024-04-14T15:17:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: osdmap e329: 8 total, 8 up, 8 in 2024-04-14T15:17:00.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: osdmap e329: 8 total, 8 up, 8 in 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: pgmap v448: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: osdmap e329: 8 total, 8 up, 8 in 2024-04-14T15:17:00.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.549 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:02.549 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:02.549 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: osdmap e330: 8 total, 8 up, 8 in 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:02.550 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:17:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: osdmap e330: 8 total, 8 up, 8 in 2024-04-14T15:17:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: pgmap v451: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi045-74490-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]': finished 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: osdmap e330: 8 total, 8 up, 8 in 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-47"}]: dispatch 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: osdmap e331: 8 total, 8 up, 8 in 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: osdmap e332: 8 total, 8 up, 8 in 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:03.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: osdmap e331: 8 total, 8 up, 8 in 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: osdmap e332: 8 total, 8 up, 8 in 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: osdmap e331: 8 total, 8 up, 8 in 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: osdmap e332: 8 total, 8 up, 8 in 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:03.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:03 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: pgmap v454: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: osdmap e333: 8 total, 8 up, 8 in 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:04 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: pgmap v454: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: osdmap e333: 8 total, 8 up, 8 in 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: pgmap v454: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: osdmap e333: 8 total, 8 up, 8 in 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3250661482' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:04.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:04 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]: dispatch 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: osdmap e334: 8 total, 8 up, 8 in 2024-04-14T15:17:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:05.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:05.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: osdmap e334: 8 total, 8 up, 8 in 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.48241 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi045-74490-62"}]': finished 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: osdmap e334: 8 total, 8 up, 8 in 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:05.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: pgmap v457: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: osdmap e335: 8 total, 8 up, 8 in 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:06 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: pgmap v457: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: osdmap e335: 8 total, 8 up, 8 in 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: pgmap v457: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi045-74490-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: osdmap e335: 8 total, 8 up, 8 in 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]: dispatch 2024-04-14T15:17:06.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:07 smithi071 ceph-mon[33150]: pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:07 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]': finished 2024-04-14T15:17:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:07 smithi071 ceph-mon[33150]: osdmap e336: 8 total, 8 up, 8 in 2024-04-14T15:17:07.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[29624]: pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:07.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]': finished 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[29624]: osdmap e336: 8 total, 8 up, 8 in 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[37795]: pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-49", "mode": "readproxy"}]': finished 2024-04-14T15:17:07.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:07 smithi045 ceph-mon[37795]: osdmap e336: 8 total, 8 up, 8 in 2024-04-14T15:17:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:09 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:09 smithi071 ceph-mon[33150]: osdmap e337: 8 total, 8 up, 8 in 2024-04-14T15:17:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:09 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:09 smithi045 ceph-mon[37795]: osdmap e337: 8 total, 8 up, 8 in 2024-04-14T15:17:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:09 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi045-74490-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:09 smithi045 ceph-mon[29624]: osdmap e337: 8 total, 8 up, 8 in 2024-04-14T15:17:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:10 smithi071 ceph-mon[33150]: pgmap v463: 236 pgs: 2 unknown, 19 creating+peering, 215 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:10 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:10 smithi071 ceph-mon[33150]: osdmap e338: 8 total, 8 up, 8 in 2024-04-14T15:17:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[37795]: pgmap v463: 236 pgs: 2 unknown, 19 creating+peering, 215 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[37795]: osdmap e338: 8 total, 8 up, 8 in 2024-04-14T15:17:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[29624]: pgmap v463: 236 pgs: 2 unknown, 19 creating+peering, 215 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:10 smithi045 ceph-mon[29624]: osdmap e338: 8 total, 8 up, 8 in 2024-04-14T15:17:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:11 smithi071 ceph-mon[33150]: osdmap e339: 8 total, 8 up, 8 in 2024-04-14T15:17:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:11 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:11.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[29624]: osdmap e339: 8 total, 8 up, 8 in 2024-04-14T15:17:11.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:11.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:11.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[37795]: osdmap e339: 8 total, 8 up, 8 in 2024-04-14T15:17:11.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:11.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:11 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:12 smithi071 ceph-mon[33150]: pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:17:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:12 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:12 smithi071 ceph-mon[33150]: osdmap e340: 8 total, 8 up, 8 in 2024-04-14T15:17:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:12 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[29624]: pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[29624]: osdmap e340: 8 total, 8 up, 8 in 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[37795]: pgmap v466: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1612426533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[37795]: osdmap e340: 8 total, 8 up, 8 in 2024-04-14T15:17:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:12 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: osdmap e341: 8 total, 8 up, 8 in 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:17:13.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:13.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: osdmap e341: 8 total, 8 up, 8 in 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.48247 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi045-74490-63"}]': finished 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: osdmap e341: 8 total, 8 up, 8 in 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: pgmap v469: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 255 B/s wr, 0 op/s 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:13.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:14.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:14 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:14 smithi071 ceph-mon[33150]: osdmap e342: 8 total, 8 up, 8 in 2024-04-14T15:17:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:14 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[29624]: osdmap e342: 8 total, 8 up, 8 in 2024-04-14T15:17:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi045-74490-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[37795]: osdmap e342: 8 total, 8 up, 8 in 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:15 smithi071 ceph-mon[33150]: osdmap e343: 8 total, 8 up, 8 in 2024-04-14T15:17:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:15 smithi071 ceph-mon[33150]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:15 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:15 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:15 smithi071 ceph-mon[33150]: osdmap e344: 8 total, 8 up, 8 in 2024-04-14T15:17:15.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[29624]: osdmap e343: 8 total, 8 up, 8 in 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[29624]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[29624]: osdmap e344: 8 total, 8 up, 8 in 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[37795]: osdmap e343: 8 total, 8 up, 8 in 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[37795]: pgmap v472: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi045-74490-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:15 smithi045 ceph-mon[37795]: osdmap e344: 8 total, 8 up, 8 in 2024-04-14T15:17:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:17.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:17 smithi071 ceph-mon[33150]: osdmap e345: 8 total, 8 up, 8 in 2024-04-14T15:17:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:17 smithi071 ceph-mon[33150]: pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:17 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:17:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[37795]: osdmap e345: 8 total, 8 up, 8 in 2024-04-14T15:17:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[37795]: pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:17:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[29624]: osdmap e345: 8 total, 8 up, 8 in 2024-04-14T15:17:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[29624]: pgmap v475: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:17 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: osdmap e346: 8 total, 8 up, 8 in 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:17:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:18 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: osdmap e346: 8 total, 8 up, 8 in 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: osdmap e346: 8 total, 8 up, 8 in 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:18.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:18.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:17:18.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:17:18.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:18 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: osdmap e347: 8 total, 8 up, 8 in 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:19 smithi071 ceph-mon[33150]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: osdmap e347: 8 total, 8 up, 8 in 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[37795]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:19.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:19.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:19.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: osdmap e347: 8 total, 8 up, 8 in 2024-04-14T15:17:19.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3199561361' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:19.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]: dispatch 2024-04-14T15:17:19.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:19 smithi045 ceph-mon[29624]: pgmap v478: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:20.321 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2163:head 2024-04-14T15:17:20.321 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-04-14T15:17:20.321 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-04-14T15:17:20.321 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-04-14T15:17:20.322 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-04-14T15:17:20.323 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8014 ms) 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: first is 1713107795 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,0 2024-04-14T15:17:20.324 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,1713107803,1713107804,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,1713107803,1713107804,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107795,1713107797,1713107798,1713107800,1713107801,1713107803,1713107804,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713107798,1713107800,1713107801,1713107803,1713107804,1713107806,1713107807,0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: first now 1713107798, trimmed 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21109 ms) 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: foo0 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14046 ms) 2024-04-14T15:17:20.325 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: osdmap e348: 8 total, 8 up, 8 in 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:20.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: osdmap e348: 8 total, 8 up, 8 in 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]': finished 2024-04-14T15:17:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.56852 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi045-74490-64"}]': finished 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: osdmap e348: 8 total, 8 up, 8 in 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-49"}]: dispatch 2024-04-14T15:17:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:21 smithi071 ceph-mon[33150]: osdmap e349: 8 total, 8 up, 8 in 2024-04-14T15:17:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:21 smithi071 ceph-mon[33150]: pgmap v481: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:21.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:21.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[37795]: osdmap e349: 8 total, 8 up, 8 in 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[37795]: pgmap v481: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi045-74490-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[29624]: osdmap e349: 8 total, 8 up, 8 in 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:21 smithi045 ceph-mon[29624]: pgmap v481: 196 pgs: 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:17:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:22 smithi071 ceph-mon[33150]: osdmap e350: 8 total, 8 up, 8 in 2024-04-14T15:17:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[29624]: osdmap e350: 8 total, 8 up, 8 in 2024-04-14T15:17:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[37795]: osdmap e350: 8 total, 8 up, 8 in 2024-04-14T15:17:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: osdmap e351: 8 total, 8 up, 8 in 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-mon[33150]: pgmap v484: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: osdmap e351: 8 total, 8 up, 8 in 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[29624]: pgmap v484: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi045-74490-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: osdmap e351: 8 total, 8 up, 8 in 2024-04-14T15:17:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:17:23.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:23 smithi045 ceph-mon[37795]: pgmap v484: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:24.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: osdmap e352: 8 total, 8 up, 8 in 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: osdmap e352: 8 total, 8 up, 8 in 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: osdmap e352: 8 total, 8 up, 8 in 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: osdmap e353: 8 total, 8 up, 8 in 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:25.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]': finished 2024-04-14T15:17:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: osdmap e353: 8 total, 8 up, 8 in 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]': finished 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: osdmap e353: 8 total, 8 up, 8 in 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]: dispatch 2024-04-14T15:17:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:25.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:25.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:17:25.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-51", "mode": "writeback"}]': finished 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: osdmap e354: 8 total, 8 up, 8 in 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:17:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:26 smithi071 ceph-mon[33150]: osdmap e355: 8 total, 8 up, 8 in 2024-04-14T15:17:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: osdmap e354: 8 total, 8 up, 8 in 2024-04-14T15:17:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[37795]: osdmap e355: 8 total, 8 up, 8 in 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: osdmap e354: 8 total, 8 up, 8 in 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]: dispatch 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2084275379' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi045-74490-65"}]': finished 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:17:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-mon[29624]: osdmap e355: 8 total, 8 up, 8 in 2024-04-14T15:17:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: pgmap v490: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: osdmap e356: 8 total, 8 up, 8 in 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:27 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: pgmap v490: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:27.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: osdmap e356: 8 total, 8 up, 8 in 2024-04-14T15:17:27.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:27.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:27.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.750 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: pgmap v490: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi045-74490-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: osdmap e356: 8 total, 8 up, 8 in 2024-04-14T15:17:27.751 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:27.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:27.752 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:27 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:17:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:17:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:17:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:28 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:17:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:17:29.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: osdmap e357: 8 total, 8 up, 8 in 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: osdmap e358: 8 total, 8 up, 8 in 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: osdmap e357: 8 total, 8 up, 8 in 2024-04-14T15:17:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: osdmap e358: 8 total, 8 up, 8 in 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:29.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: osdmap e357: 8 total, 8 up, 8 in 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi045-74490-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: osdmap e358: 8 total, 8 up, 8 in 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:17:30.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[29624]: osdmap e359: 8 total, 8 up, 8 in 2024-04-14T15:17:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:17:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:30 smithi045 ceph-mon[37795]: osdmap e359: 8 total, 8 up, 8 in 2024-04-14T15:17:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:30 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-51","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:17:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:30 smithi071 ceph-mon[33150]: osdmap e359: 8 total, 8 up, 8 in 2024-04-14T15:17:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[37795]: pgmap v496: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[37795]: osdmap e360: 8 total, 8 up, 8 in 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[29624]: pgmap v496: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[29624]: osdmap e360: 8 total, 8 up, 8 in 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:31 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:31 smithi071 ceph-mon[33150]: pgmap v496: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:17:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:31 smithi071 ceph-mon[33150]: osdmap e360: 8 total, 8 up, 8 in 2024-04-14T15:17:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:31 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[37795]: osdmap e361: 8 total, 8 up, 8 in 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:17:32.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:32.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[29624]: osdmap e361: 8 total, 8 up, 8 in 2024-04-14T15:17:32.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:32 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:32 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:17:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:32 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:32 smithi071 ceph-mon[33150]: osdmap e361: 8 total, 8 up, 8 in 2024-04-14T15:17:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3503820918' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:32 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]: dispatch 2024-04-14T15:17:33.364 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ 2024-04-14T15:17:33.364 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3028 ms) 2024-04-14T15:17:33.364 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-04-14T15:17:33.364 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7011 ms) 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7012 ms) 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7024 ms) 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7038 ms) 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-04-14T15:17:33.365 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7031 ms) 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7012 ms) 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139316 ms total) 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-04-14T15:17:33.366 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (289945 ms total) 2024-04-14T15:17:33.367 INFO:tasks.workunit.client.0.smithi045.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-04-14T15:17:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[29624]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:17:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[29624]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[29624]: osdmap e362: 8 total, 8 up, 8 in 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[37795]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[37795]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:33 smithi045 ceph-mon[37795]: osdmap e362: 8 total, 8 up, 8 in 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:33 smithi071 ceph-mon[33150]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:33 smithi071 ceph-mon[33150]: from='client.56864 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi045-74490-66"}]': finished 2024-04-14T15:17:33.747 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:33 smithi071 ceph-mon[33150]: osdmap e362: 8 total, 8 up, 8 in 2024-04-14T15:17:34.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:34.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:34.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:34.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:35 smithi045 ceph-mon[37795]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:35 smithi045 ceph-mon[29624]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:35 smithi071 ceph-mon[33150]: pgmap v501: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:37 smithi071 ceph-mon[33150]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:37.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:37 smithi045 ceph-mon[37795]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:37.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:37 smithi045 ceph-mon[29624]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:39.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:39 smithi071 ceph-mon[33150]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:39.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:39 smithi045 ceph-mon[29624]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:39.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:39 smithi045 ceph-mon[37795]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:41 smithi071 ceph-mon[33150]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:41 smithi071 ceph-mon[33150]: osdmap e363: 8 total, 8 up, 8 in 2024-04-14T15:17:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:41.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[29624]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:41.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:41.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[29624]: osdmap e363: 8 total, 8 up, 8 in 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[37795]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[37795]: osdmap e363: 8 total, 8 up, 8 in 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:41.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: osdmap e364: 8 total, 8 up, 8 in 2024-04-14T15:17:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:42.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: osdmap e364: 8 total, 8 up, 8 in 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]': finished 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: osdmap e364: 8 total, 8 up, 8 in 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:42.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:42.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-51"}]: dispatch 2024-04-14T15:17:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-mon[33150]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:17:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:17:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-mon[33150]: osdmap e365: 8 total, 8 up, 8 in 2024-04-14T15:17:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:43.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:43.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[29624]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:17:43.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:17:43.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[29624]: osdmap e365: 8 total, 8 up, 8 in 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[37795]: pgmap v507: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[37795]: osdmap e365: 8 total, 8 up, 8 in 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:17:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:44 smithi071 ceph-mon[33150]: osdmap e366: 8 total, 8 up, 8 in 2024-04-14T15:17:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[29624]: osdmap e366: 8 total, 8 up, 8 in 2024-04-14T15:17:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[37795]: osdmap e366: 8 total, 8 up, 8 in 2024-04-14T15:17:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: osdmap e367: 8 total, 8 up, 8 in 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: osdmap e367: 8 total, 8 up, 8 in 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:45.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: pgmap v510: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: osdmap e367: 8 total, 8 up, 8 in 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:45.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-53"}]: dispatch 2024-04-14T15:17:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:46 smithi071 ceph-mon[33150]: osdmap e368: 8 total, 8 up, 8 in 2024-04-14T15:17:46.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:46 smithi045 ceph-mon[37795]: osdmap e368: 8 total, 8 up, 8 in 2024-04-14T15:17:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:46 smithi045 ceph-mon[29624]: osdmap e368: 8 total, 8 up, 8 in 2024-04-14T15:17:46.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:47 smithi071 ceph-mon[33150]: pgmap v513: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:47 smithi071 ceph-mon[33150]: osdmap e369: 8 total, 8 up, 8 in 2024-04-14T15:17:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:47.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[37795]: pgmap v513: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:47.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[37795]: osdmap e369: 8 total, 8 up, 8 in 2024-04-14T15:17:47.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:47.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:47.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[29624]: pgmap v513: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:17:47.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[29624]: osdmap e369: 8 total, 8 up, 8 in 2024-04-14T15:17:47.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:47.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: osdmap e370: 8 total, 8 up, 8 in 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:48.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:48.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:48.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: osdmap e370: 8 total, 8 up, 8 in 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: osdmap e370: 8 total, 8 up, 8 in 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:48.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-55"}]: dispatch 2024-04-14T15:17:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:49 smithi071 ceph-mon[33150]: pgmap v516: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-14T15:17:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:49 smithi071 ceph-mon[33150]: osdmap e371: 8 total, 8 up, 8 in 2024-04-14T15:17:49.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:49 smithi045 ceph-mon[37795]: pgmap v516: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-14T15:17:49.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:49 smithi045 ceph-mon[37795]: osdmap e371: 8 total, 8 up, 8 in 2024-04-14T15:17:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:49 smithi045 ceph-mon[29624]: pgmap v516: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-04-14T15:17:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:49 smithi045 ceph-mon[29624]: osdmap e371: 8 total, 8 up, 8 in 2024-04-14T15:17:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:50 smithi071 ceph-mon[33150]: osdmap e372: 8 total, 8 up, 8 in 2024-04-14T15:17:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:50 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:50.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[37795]: osdmap e372: 8 total, 8 up, 8 in 2024-04-14T15:17:50.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[29624]: osdmap e372: 8 total, 8 up, 8 in 2024-04-14T15:17:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:50 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: pgmap v519: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: osdmap e373: 8 total, 8 up, 8 in 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: pgmap v519: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: osdmap e373: 8 total, 8 up, 8 in 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: pgmap v519: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: osdmap e373: 8 total, 8 up, 8 in 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:51.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:51.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-57"}]: dispatch 2024-04-14T15:17:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:52 smithi071 ceph-mon[33150]: osdmap e374: 8 total, 8 up, 8 in 2024-04-14T15:17:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:52 smithi045 ceph-mon[37795]: osdmap e374: 8 total, 8 up, 8 in 2024-04-14T15:17:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:52 smithi045 ceph-mon[29624]: osdmap e374: 8 total, 8 up, 8 in 2024-04-14T15:17:53.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[29624]: pgmap v522: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[29624]: osdmap e375: 8 total, 8 up, 8 in 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[37795]: pgmap v522: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[37795]: osdmap e375: 8 total, 8 up, 8 in 2024-04-14T15:17:53.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:53.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:53 smithi071 ceph-mon[33150]: pgmap v522: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:17:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:53 smithi071 ceph-mon[33150]: osdmap e375: 8 total, 8 up, 8 in 2024-04-14T15:17:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:54.108 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:17:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:17:54.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:54.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: osdmap e376: 8 total, 8 up, 8 in 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: osdmap e376: 8 total, 8 up, 8 in 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:54.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:54.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:54.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:54.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: osdmap e376: 8 total, 8 up, 8 in 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-59"}]: dispatch 2024-04-14T15:17:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:17:55.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:55 smithi045 ceph-mon[37795]: pgmap v525: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:55.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:55 smithi045 ceph-mon[37795]: osdmap e377: 8 total, 8 up, 8 in 2024-04-14T15:17:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:55 smithi045 ceph-mon[29624]: pgmap v525: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:55 smithi045 ceph-mon[29624]: osdmap e377: 8 total, 8 up, 8 in 2024-04-14T15:17:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:55 smithi071 ceph-mon[33150]: pgmap v525: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:55 smithi071 ceph-mon[33150]: osdmap e377: 8 total, 8 up, 8 in 2024-04-14T15:17:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:17:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[37795]: osdmap e378: 8 total, 8 up, 8 in 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[29624]: osdmap e378: 8 total, 8 up, 8 in 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:56.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:57.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:56 smithi071 ceph-mon[33150]: osdmap e378: 8 total, 8 up, 8 in 2024-04-14T15:17:57.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:57.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:17:57.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: osdmap e379: 8 total, 8 up, 8 in 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:57.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: osdmap e379: 8 total, 8 up, 8 in 2024-04-14T15:17:57.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:57.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:57.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:57 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: pgmap v528: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: osdmap e379: 8 total, 8 up, 8 in 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:57 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: osdmap e380: 8 total, 8 up, 8 in 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:17:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: osdmap e380: 8 total, 8 up, 8 in 2024-04-14T15:17:58.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:58.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:58.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:58.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:58.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: osdmap e380: 8 total, 8 up, 8 in 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-61"}]: dispatch 2024-04-14T15:17:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:17:59.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:59 smithi045 ceph-mon[37795]: pgmap v531: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:17:59.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:17:59 smithi045 ceph-mon[37795]: osdmap e381: 8 total, 8 up, 8 in 2024-04-14T15:17:59.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:59 smithi045 ceph-mon[29624]: pgmap v531: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:17:59.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:17:59 smithi045 ceph-mon[29624]: osdmap e381: 8 total, 8 up, 8 in 2024-04-14T15:18:00.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:59 smithi071 ceph-mon[33150]: pgmap v531: 228 pgs: 9 unknown, 219 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:18:00.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:17:59 smithi071 ceph-mon[33150]: osdmap e381: 8 total, 8 up, 8 in 2024-04-14T15:18:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[37795]: osdmap e382: 8 total, 8 up, 8 in 2024-04-14T15:18:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:00.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:00.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[29624]: osdmap e382: 8 total, 8 up, 8 in 2024-04-14T15:18:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:00.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:00 smithi071 ceph-mon[33150]: osdmap e382: 8 total, 8 up, 8 in 2024-04-14T15:18:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: osdmap e383: 8 total, 8 up, 8 in 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:01.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: osdmap e383: 8 total, 8 up, 8 in 2024-04-14T15:18:01.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:01.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:01.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: osdmap e383: 8 total, 8 up, 8 in 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:02.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:02.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:02 smithi045 ceph-mon[29624]: osdmap e384: 8 total, 8 up, 8 in 2024-04-14T15:18:02.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:02.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:02 smithi045 ceph-mon[37795]: osdmap e384: 8 total, 8 up, 8 in 2024-04-14T15:18:03.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:03.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:02 smithi071 ceph-mon[33150]: osdmap e384: 8 total, 8 up, 8 in 2024-04-14T15:18:03.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[29624]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:03.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:03.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[29624]: osdmap e385: 8 total, 8 up, 8 in 2024-04-14T15:18:03.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[37795]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:03.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:03.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:03 smithi045 ceph-mon[37795]: osdmap e385: 8 total, 8 up, 8 in 2024-04-14T15:18:04.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:03 smithi071 ceph-mon[33150]: pgmap v537: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:04.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:03 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:04.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:03 smithi071 ceph-mon[33150]: osdmap e385: 8 total, 8 up, 8 in 2024-04-14T15:18:04.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:04.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:04 smithi045 ceph-mon[37795]: osdmap e386: 8 total, 8 up, 8 in 2024-04-14T15:18:04.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:04 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:04.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:04 smithi045 ceph-mon[29624]: osdmap e386: 8 total, 8 up, 8 in 2024-04-14T15:18:04.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:04 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:04 smithi071 ceph-mon[33150]: osdmap e386: 8 total, 8 up, 8 in 2024-04-14T15:18:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:04 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:05 smithi045 ceph-mon[37795]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:05 smithi045 ceph-mon[37795]: osdmap e387: 8 total, 8 up, 8 in 2024-04-14T15:18:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:05 smithi045 ceph-mon[29624]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:05 smithi045 ceph-mon[29624]: osdmap e387: 8 total, 8 up, 8 in 2024-04-14T15:18:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:05 smithi071 ceph-mon[33150]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:05 smithi071 ceph-mon[33150]: osdmap e387: 8 total, 8 up, 8 in 2024-04-14T15:18:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:06 smithi045 ceph-mon[29624]: osdmap e388: 8 total, 8 up, 8 in 2024-04-14T15:18:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:06 smithi045 ceph-mon[37795]: osdmap e388: 8 total, 8 up, 8 in 2024-04-14T15:18:07.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:06 smithi071 ceph-mon[33150]: osdmap e388: 8 total, 8 up, 8 in 2024-04-14T15:18:07.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:07 smithi045 ceph-mon[37795]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:07.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:07 smithi045 ceph-mon[29624]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:07 smithi071 ceph-mon[33150]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-04-14T15:18:09.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:09 smithi045 ceph-mon[37795]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-14T15:18:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:09 smithi045 ceph-mon[29624]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-14T15:18:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:09 smithi071 ceph-mon[33150]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-04-14T15:18:11.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:11 smithi045 ceph-mon[37795]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 601 B/s wr, 2 op/s 2024-04-14T15:18:11.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:11 smithi045 ceph-mon[29624]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 601 B/s wr, 2 op/s 2024-04-14T15:18:12.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:11 smithi071 ceph-mon[33150]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 601 B/s wr, 2 op/s 2024-04-14T15:18:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:13 smithi045 ceph-mon[29624]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:18:13.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:13 smithi045 ceph-mon[37795]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:18:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:13 smithi071 ceph-mon[33150]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:18:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:14.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:14.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:14.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:16 smithi071 ceph-mon[33150]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 418 B/s wr, 2 op/s 2024-04-14T15:18:16.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:16 smithi071 ceph-mon[33150]: osdmap e389: 8 total, 8 up, 8 in 2024-04-14T15:18:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:16 smithi045 ceph-mon[37795]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 418 B/s wr, 2 op/s 2024-04-14T15:18:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:16 smithi045 ceph-mon[37795]: osdmap e389: 8 total, 8 up, 8 in 2024-04-14T15:18:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:16 smithi045 ceph-mon[29624]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 418 B/s wr, 2 op/s 2024-04-14T15:18:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:16 smithi045 ceph-mon[29624]: osdmap e389: 8 total, 8 up, 8 in 2024-04-14T15:18:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:17.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:17 smithi071 ceph-mon[33150]: osdmap e390: 8 total, 8 up, 8 in 2024-04-14T15:18:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:17 smithi045 ceph-mon[37795]: osdmap e390: 8 total, 8 up, 8 in 2024-04-14T15:18:17.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:17 smithi045 ceph-mon[29624]: osdmap e390: 8 total, 8 up, 8 in 2024-04-14T15:18:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:18 smithi071 ceph-mon[33150]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:18.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:18 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:18:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:18 smithi045 ceph-mon[37795]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:18 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:18:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:18 smithi045 ceph-mon[29624]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:18 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:18:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:18:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:18:19.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:18:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:18:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:18:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:18:19.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:18:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:18:19.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:18:20.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:20 smithi071 ceph-mon[33150]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:20 smithi045 ceph-mon[37795]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:20 smithi045 ceph-mon[29624]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:22.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:22 smithi071 ceph-mon[33150]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:22 smithi045 ceph-mon[37795]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:22 smithi045 ceph-mon[29624]: pgmap v552: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:24.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:24.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:24 smithi071 ceph-mon[33150]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:24 smithi045 ceph-mon[29624]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:24 smithi045 ceph-mon[37795]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:25.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:25 smithi071 ceph-mon[33150]: osdmap e391: 8 total, 8 up, 8 in 2024-04-14T15:18:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:25 smithi045 ceph-mon[29624]: osdmap e391: 8 total, 8 up, 8 in 2024-04-14T15:18:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:25.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:25 smithi045 ceph-mon[37795]: osdmap e391: 8 total, 8 up, 8 in 2024-04-14T15:18:26.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:26 smithi071 ceph-mon[33150]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:26 smithi045 ceph-mon[29624]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:26 smithi045 ceph-mon[37795]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:18:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:27.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:27 smithi071 ceph-mon[33150]: osdmap e392: 8 total, 8 up, 8 in 2024-04-14T15:18:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:27 smithi045 ceph-mon[37795]: osdmap e392: 8 total, 8 up, 8 in 2024-04-14T15:18:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:27 smithi045 ceph-mon[29624]: osdmap e392: 8 total, 8 up, 8 in 2024-04-14T15:18:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:28 smithi071 ceph-mon[33150]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:28.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:28 smithi045 ceph-mon[37795]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:28 smithi045 ceph-mon[29624]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:18:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:30.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:30 smithi071 ceph-mon[33150]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:30 smithi045 ceph-mon[37795]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:30 smithi045 ceph-mon[29624]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:32 smithi071 ceph-mon[33150]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:32 smithi045 ceph-mon[37795]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:32 smithi045 ceph-mon[29624]: pgmap v559: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:18:34.104 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:34 smithi071 ceph-mon[33150]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-04-14T15:18:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:34 smithi045 ceph-mon[37795]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-04-14T15:18:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:34 smithi045 ceph-mon[29624]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-04-14T15:18:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:35 smithi045 ceph-mon[37795]: osdmap e393: 8 total, 8 up, 8 in 2024-04-14T15:18:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:35.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:35 smithi045 ceph-mon[29624]: osdmap e393: 8 total, 8 up, 8 in 2024-04-14T15:18:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:35 smithi071 ceph-mon[33150]: osdmap e393: 8 total, 8 up, 8 in 2024-04-14T15:18:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[29624]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[37795]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:36 smithi071 ceph-mon[33150]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:18:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-63"}]: dispatch 2024-04-14T15:18:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:38.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:38 smithi045 ceph-mon[37795]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:18:38.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:38 smithi045 ceph-mon[37795]: osdmap e394: 8 total, 8 up, 8 in 2024-04-14T15:18:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:38 smithi045 ceph-mon[29624]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:18:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:38 smithi045 ceph-mon[29624]: osdmap e394: 8 total, 8 up, 8 in 2024-04-14T15:18:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:38 smithi071 ceph-mon[33150]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:18:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:38 smithi071 ceph-mon[33150]: osdmap e394: 8 total, 8 up, 8 in 2024-04-14T15:18:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[37795]: osdmap e395: 8 total, 8 up, 8 in 2024-04-14T15:18:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:39.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[29624]: osdmap e395: 8 total, 8 up, 8 in 2024-04-14T15:18:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:39 smithi071 ceph-mon[33150]: osdmap e395: 8 total, 8 up, 8 in 2024-04-14T15:18:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: pgmap v566: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: osdmap e396: 8 total, 8 up, 8 in 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: pgmap v566: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: osdmap e396: 8 total, 8 up, 8 in 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: pgmap v566: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: osdmap e396: 8 total, 8 up, 8 in 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:18:41.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:41.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:41 smithi045 ceph-mon[29624]: osdmap e397: 8 total, 8 up, 8 in 2024-04-14T15:18:41.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:41.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:41 smithi045 ceph-mon[37795]: osdmap e397: 8 total, 8 up, 8 in 2024-04-14T15:18:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:18:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:41 smithi071 ceph-mon[33150]: osdmap e397: 8 total, 8 up, 8 in 2024-04-14T15:18:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:42 smithi045 ceph-mon[29624]: pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:42.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:42 smithi045 ceph-mon[29624]: osdmap e398: 8 total, 8 up, 8 in 2024-04-14T15:18:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:42 smithi045 ceph-mon[37795]: pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:42.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:42 smithi045 ceph-mon[37795]: osdmap e398: 8 total, 8 up, 8 in 2024-04-14T15:18:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:42 smithi071 ceph-mon[33150]: pgmap v569: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:42 smithi071 ceph-mon[33150]: osdmap e398: 8 total, 8 up, 8 in 2024-04-14T15:18:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:43 smithi045 ceph-mon[37795]: osdmap e399: 8 total, 8 up, 8 in 2024-04-14T15:18:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:43.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:43 smithi045 ceph-mon[29624]: osdmap e399: 8 total, 8 up, 8 in 2024-04-14T15:18:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:43 smithi071 ceph-mon[33150]: osdmap e399: 8 total, 8 up, 8 in 2024-04-14T15:18:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:44.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:44 smithi045 ceph-mon[37795]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:44 smithi045 ceph-mon[37795]: osdmap e400: 8 total, 8 up, 8 in 2024-04-14T15:18:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:44 smithi045 ceph-mon[29624]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:44 smithi045 ceph-mon[29624]: osdmap e400: 8 total, 8 up, 8 in 2024-04-14T15:18:44.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:44 smithi071 ceph-mon[33150]: pgmap v572: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:44 smithi071 ceph-mon[33150]: osdmap e400: 8 total, 8 up, 8 in 2024-04-14T15:18:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:45.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:45 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:45.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:45 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:45 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:18:46.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:46 smithi045 ceph-mon[37795]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-14T15:18:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:46 smithi045 ceph-mon[29624]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-14T15:18:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:46 smithi071 ceph-mon[33150]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-14T15:18:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:48.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:48 smithi045 ceph-mon[37795]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-14T15:18:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:48 smithi045 ceph-mon[29624]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-14T15:18:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:48 smithi071 ceph-mon[33150]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-14T15:18:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:50 smithi045 ceph-mon[29624]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-14T15:18:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:50 smithi045 ceph-mon[29624]: osdmap e401: 8 total, 8 up, 8 in 2024-04-14T15:18:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:50 smithi045 ceph-mon[37795]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-14T15:18:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:50 smithi045 ceph-mon[37795]: osdmap e401: 8 total, 8 up, 8 in 2024-04-14T15:18:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:50 smithi071 ceph-mon[33150]: pgmap v576: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-14T15:18:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:50 smithi071 ceph-mon[33150]: osdmap e401: 8 total, 8 up, 8 in 2024-04-14T15:18:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:52 smithi045 ceph-mon[29624]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-14T15:18:52.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:52 smithi045 ceph-mon[37795]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-14T15:18:52.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:52 smithi071 ceph-mon[33150]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-04-14T15:18:54.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:18:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:18:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[37795]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-14T15:18:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:54.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[29624]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-14T15:18:54.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:54.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:54 smithi071 ceph-mon[33150]: pgmap v579: 228 pgs: 228 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-04-14T15:18:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-65"}]: dispatch 2024-04-14T15:18:55.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:55.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:55 smithi045 ceph-mon[37795]: osdmap e402: 8 total, 8 up, 8 in 2024-04-14T15:18:55.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:55 smithi045 ceph-mon[29624]: osdmap e402: 8 total, 8 up, 8 in 2024-04-14T15:18:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:18:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:55 smithi071 ceph-mon[33150]: osdmap e402: 8 total, 8 up, 8 in 2024-04-14T15:18:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[29624]: pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[29624]: osdmap e403: 8 total, 8 up, 8 in 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[37795]: pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[37795]: osdmap e403: 8 total, 8 up, 8 in 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:56 smithi071 ceph-mon[33150]: pgmap v581: 196 pgs: 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:56.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:56 smithi071 ceph-mon[33150]: osdmap e403: 8 total, 8 up, 8 in 2024-04-14T15:18:56.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:18:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:18:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:18:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:18:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[37795]: osdmap e404: 8 total, 8 up, 8 in 2024-04-14T15:18:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[29624]: osdmap e404: 8 total, 8 up, 8 in 2024-04-14T15:18:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:57 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:18:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:57 smithi071 ceph-mon[33150]: osdmap e404: 8 total, 8 up, 8 in 2024-04-14T15:18:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:18:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: pgmap v584: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: osdmap e405: 8 total, 8 up, 8 in 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: pgmap v584: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: osdmap e405: 8 total, 8 up, 8 in 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: pgmap v584: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: osdmap e405: 8 total, 8 up, 8 in 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-67"}]: dispatch 2024-04-14T15:18:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:18:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:18:59 smithi045 ceph-mon[29624]: osdmap e406: 8 total, 8 up, 8 in 2024-04-14T15:18:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:18:59 smithi045 ceph-mon[37795]: osdmap e406: 8 total, 8 up, 8 in 2024-04-14T15:18:59.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:18:59 smithi071 ceph-mon[33150]: osdmap e406: 8 total, 8 up, 8 in 2024-04-14T15:19:00.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[29624]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:00.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[29624]: osdmap e407: 8 total, 8 up, 8 in 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[37795]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[37795]: osdmap e407: 8 total, 8 up, 8 in 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:00 smithi071 ceph-mon[33150]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:00 smithi071 ceph-mon[33150]: osdmap e407: 8 total, 8 up, 8 in 2024-04-14T15:19:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: osdmap e408: 8 total, 8 up, 8 in 2024-04-14T15:19:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: osdmap e408: 8 total, 8 up, 8 in 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: osdmap e408: 8 total, 8 up, 8 in 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-69"}]: dispatch 2024-04-14T15:19:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:02 smithi045 ceph-mon[37795]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:19:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:02 smithi045 ceph-mon[37795]: osdmap e409: 8 total, 8 up, 8 in 2024-04-14T15:19:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:02 smithi045 ceph-mon[29624]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:19:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:02 smithi045 ceph-mon[29624]: osdmap e409: 8 total, 8 up, 8 in 2024-04-14T15:19:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:02 smithi071 ceph-mon[33150]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:19:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:02 smithi071 ceph-mon[33150]: osdmap e409: 8 total, 8 up, 8 in 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[29624]: osdmap e410: 8 total, 8 up, 8 in 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[37795]: osdmap e410: 8 total, 8 up, 8 in 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:03 smithi071 ceph-mon[33150]: osdmap e410: 8 total, 8 up, 8 in 2024-04-14T15:19:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:04.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 511 B/s wr, 0 op/s 2024-04-14T15:19:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: osdmap e411: 8 total, 8 up, 8 in 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 511 B/s wr, 0 op/s 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: osdmap e411: 8 total, 8 up, 8 in 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 511 B/s wr, 0 op/s 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: osdmap e411: 8 total, 8 up, 8 in 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:19:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:19:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:05 smithi071 ceph-mon[33150]: osdmap e412: 8 total, 8 up, 8 in 2024-04-14T15:19:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:19:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[37795]: osdmap e412: 8 total, 8 up, 8 in 2024-04-14T15:19:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:19:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:05 smithi045 ceph-mon[29624]: osdmap e412: 8 total, 8 up, 8 in 2024-04-14T15:19:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:06 smithi071 ceph-mon[33150]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:19:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:06 smithi071 ceph-mon[33150]: osdmap e413: 8 total, 8 up, 8 in 2024-04-14T15:19:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:06 smithi045 ceph-mon[29624]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:19:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:06 smithi045 ceph-mon[29624]: osdmap e413: 8 total, 8 up, 8 in 2024-04-14T15:19:06.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:06 smithi045 ceph-mon[37795]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:19:06.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:06 smithi045 ceph-mon[37795]: osdmap e413: 8 total, 8 up, 8 in 2024-04-14T15:19:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:08 smithi071 ceph-mon[33150]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-04-14T15:19:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:08 smithi045 ceph-mon[29624]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-04-14T15:19:08.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:08 smithi045 ceph-mon[37795]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-04-14T15:19:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:10 smithi071 ceph-mon[33150]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:10 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:10 smithi045 ceph-mon[37795]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:10 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:10 smithi045 ceph-mon[29624]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:10 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:12 smithi071 ceph-mon[33150]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-14T15:19:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:12 smithi045 ceph-mon[29624]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-14T15:19:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:12 smithi045 ceph-mon[37795]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-04-14T15:19:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:14.106 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:14 smithi071 ceph-mon[33150]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 743 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-14T15:19:14.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:14 smithi045 ceph-mon[37795]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 743 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-14T15:19:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:14 smithi045 ceph-mon[29624]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 743 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-14T15:19:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:16 smithi071 ceph-mon[33150]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:19:16.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:16 smithi045 ceph-mon[37795]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:19:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:16 smithi045 ceph-mon[29624]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:19:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:18 smithi071 ceph-mon[33150]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1000 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:18 smithi045 ceph-mon[37795]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1000 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:18 smithi045 ceph-mon[29624]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1000 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:19:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:19:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:19:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:19:19.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:19:19.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:19:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:19:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:20 smithi071 ceph-mon[33150]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:20.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:20 smithi045 ceph-mon[29624]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:20 smithi045 ceph-mon[37795]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-04-14T15:19:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:22 smithi071 ceph-mon[33150]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-14T15:19:22.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:22 smithi045 ceph-mon[37795]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-14T15:19:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:22 smithi045 ceph-mon[29624]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-14T15:19:24.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:24 smithi071 ceph-mon[33150]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:24 smithi045 ceph-mon[37795]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:24 smithi045 ceph-mon[29624]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:25.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:26 smithi071 ceph-mon[33150]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:26 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[29624]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:26.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[37795]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-71"}]: dispatch 2024-04-14T15:19:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:27 smithi071 ceph-mon[33150]: osdmap e414: 8 total, 8 up, 8 in 2024-04-14T15:19:27.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:27 smithi045 ceph-mon[37795]: osdmap e414: 8 total, 8 up, 8 in 2024-04-14T15:19:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:27 smithi045 ceph-mon[29624]: osdmap e414: 8 total, 8 up, 8 in 2024-04-14T15:19:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:28 smithi071 ceph-mon[33150]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-04-14T15:19:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:28 smithi071 ceph-mon[33150]: osdmap e415: 8 total, 8 up, 8 in 2024-04-14T15:19:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[37795]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-04-14T15:19:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[37795]: osdmap e415: 8 total, 8 up, 8 in 2024-04-14T15:19:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[29624]: pgmap v609: 196 pgs: 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[29624]: osdmap e415: 8 total, 8 up, 8 in 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:29 smithi071 ceph-mon[33150]: osdmap e416: 8 total, 8 up, 8 in 2024-04-14T15:19:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:29 smithi071 ceph-mon[33150]: pgmap v612: 228 pgs: 5 unknown, 223 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:29.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:29.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[29624]: osdmap e416: 8 total, 8 up, 8 in 2024-04-14T15:19:29.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[29624]: pgmap v612: 228 pgs: 5 unknown, 223 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[37795]: osdmap e416: 8 total, 8 up, 8 in 2024-04-14T15:19:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:29 smithi045 ceph-mon[37795]: pgmap v612: 228 pgs: 5 unknown, 223 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:30 smithi071 ceph-mon[33150]: osdmap e417: 8 total, 8 up, 8 in 2024-04-14T15:19:30.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:30 smithi045 ceph-mon[29624]: osdmap e417: 8 total, 8 up, 8 in 2024-04-14T15:19:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:30 smithi045 ceph-mon[37795]: osdmap e417: 8 total, 8 up, 8 in 2024-04-14T15:19:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:31 smithi071 ceph-mon[33150]: osdmap e418: 8 total, 8 up, 8 in 2024-04-14T15:19:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:31 smithi071 ceph-mon[33150]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:31 smithi045 ceph-mon[37795]: osdmap e418: 8 total, 8 up, 8 in 2024-04-14T15:19:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:31 smithi045 ceph-mon[37795]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:31 smithi045 ceph-mon[29624]: osdmap e418: 8 total, 8 up, 8 in 2024-04-14T15:19:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:31 smithi045 ceph-mon[29624]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:33.853 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:33 smithi071 ceph-mon[33150]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 983 B/s wr, 1 op/s 2024-04-14T15:19:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:33 smithi045 ceph-mon[29624]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 983 B/s wr, 1 op/s 2024-04-14T15:19:33.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:33 smithi045 ceph-mon[37795]: pgmap v616: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 983 B/s rd, 983 B/s wr, 1 op/s 2024-04-14T15:19:34.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:34.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:34.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:34.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:35 smithi071 ceph-mon[33150]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-14T15:19:35.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:35 smithi045 ceph-mon[37795]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-14T15:19:35.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:35 smithi045 ceph-mon[29624]: pgmap v617: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-14T15:19:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:37 smithi071 ceph-mon[33150]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s 2024-04-14T15:19:37.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:37 smithi045 ceph-mon[29624]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s 2024-04-14T15:19:37.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:37 smithi045 ceph-mon[37795]: pgmap v618: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 383 B/s wr, 2 op/s 2024-04-14T15:19:39.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:39 smithi071 ceph-mon[33150]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 222 B/s wr, 2 op/s 2024-04-14T15:19:39.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:39 smithi045 ceph-mon[29624]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 222 B/s wr, 2 op/s 2024-04-14T15:19:39.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:39 smithi045 ceph-mon[37795]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 222 B/s wr, 2 op/s 2024-04-14T15:19:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:40.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:40.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:40.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-73"}]: dispatch 2024-04-14T15:19:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:41 smithi071 ceph-mon[33150]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-14T15:19:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:41 smithi071 ceph-mon[33150]: osdmap e419: 8 total, 8 up, 8 in 2024-04-14T15:19:41.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:41 smithi045 ceph-mon[29624]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-14T15:19:41.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:41 smithi045 ceph-mon[29624]: osdmap e419: 8 total, 8 up, 8 in 2024-04-14T15:19:41.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:41 smithi045 ceph-mon[37795]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-14T15:19:41.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:41 smithi045 ceph-mon[37795]: osdmap e419: 8 total, 8 up, 8 in 2024-04-14T15:19:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:42 smithi071 ceph-mon[33150]: osdmap e420: 8 total, 8 up, 8 in 2024-04-14T15:19:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:42 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:42.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[29624]: osdmap e420: 8 total, 8 up, 8 in 2024-04-14T15:19:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:42.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[37795]: osdmap e420: 8 total, 8 up, 8 in 2024-04-14T15:19:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:42.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:42 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: osdmap e421: 8 total, 8 up, 8 in 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:43.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: osdmap e421: 8 total, 8 up, 8 in 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: pgmap v623: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: osdmap e421: 8 total, 8 up, 8 in 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:43.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:44.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:44 smithi071 ceph-mon[33150]: osdmap e422: 8 total, 8 up, 8 in 2024-04-14T15:19:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]: dispatch 2024-04-14T15:19:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[29624]: osdmap e422: 8 total, 8 up, 8 in 2024-04-14T15:19:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:44.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]: dispatch 2024-04-14T15:19:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[37795]: osdmap e422: 8 total, 8 up, 8 in 2024-04-14T15:19:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:44.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]: dispatch 2024-04-14T15:19:45.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[29624]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:45.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]': finished 2024-04-14T15:19:45.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[29624]: osdmap e423: 8 total, 8 up, 8 in 2024-04-14T15:19:45.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[37795]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:45.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]': finished 2024-04-14T15:19:45.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:45 smithi045 ceph-mon[37795]: osdmap e423: 8 total, 8 up, 8 in 2024-04-14T15:19:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:45 smithi071 ceph-mon[33150]: pgmap v626: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:45 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "303.18", "id": [0, 4]}]': finished 2024-04-14T15:19:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:45 smithi071 ceph-mon[33150]: osdmap e423: 8 total, 8 up, 8 in 2024-04-14T15:19:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[29624]: osdmap e424: 8 total, 8 up, 8 in 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[37795]: osdmap e424: 8 total, 8 up, 8 in 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:46 smithi071 ceph-mon[33150]: osdmap e424: 8 total, 8 up, 8 in 2024-04-14T15:19:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-75"}]: dispatch 2024-04-14T15:19:47.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:47 smithi045 ceph-mon[29624]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:47.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:47 smithi045 ceph-mon[29624]: osdmap e425: 8 total, 8 up, 8 in 2024-04-14T15:19:47.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:47 smithi045 ceph-mon[37795]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:47.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:47 smithi045 ceph-mon[37795]: osdmap e425: 8 total, 8 up, 8 in 2024-04-14T15:19:48.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:47 smithi071 ceph-mon[33150]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:19:48.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:47 smithi071 ceph-mon[33150]: osdmap e425: 8 total, 8 up, 8 in 2024-04-14T15:19:48.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[29624]: osdmap e426: 8 total, 8 up, 8 in 2024-04-14T15:19:48.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[37795]: osdmap e426: 8 total, 8 up, 8 in 2024-04-14T15:19:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:48 smithi071 ceph-mon[33150]: osdmap e426: 8 total, 8 up, 8 in 2024-04-14T15:19:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[29624]: pgmap v632: 228 pgs: 24 creating+peering, 8 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[29624]: osdmap e427: 8 total, 8 up, 8 in 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[37795]: pgmap v632: 228 pgs: 24 creating+peering, 8 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[37795]: osdmap e427: 8 total, 8 up, 8 in 2024-04-14T15:19:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:49 smithi071 ceph-mon[33150]: pgmap v632: 228 pgs: 24 creating+peering, 8 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:49 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:49 smithi071 ceph-mon[33150]: osdmap e427: 8 total, 8 up, 8 in 2024-04-14T15:19:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:50.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[29624]: osdmap e428: 8 total, 8 up, 8 in 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[37795]: osdmap e428: 8 total, 8 up, 8 in 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:51.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:50 smithi071 ceph-mon[33150]: osdmap e428: 8 total, 8 up, 8 in 2024-04-14T15:19:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:19:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-77"}]: dispatch 2024-04-14T15:19:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:51 smithi045 ceph-mon[37795]: pgmap v635: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:51.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:51 smithi045 ceph-mon[37795]: osdmap e429: 8 total, 8 up, 8 in 2024-04-14T15:19:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:51 smithi045 ceph-mon[29624]: pgmap v635: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:51 smithi045 ceph-mon[29624]: osdmap e429: 8 total, 8 up, 8 in 2024-04-14T15:19:52.008 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:51 smithi071 ceph-mon[33150]: pgmap v635: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:19:52.008 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:51 smithi071 ceph-mon[33150]: osdmap e429: 8 total, 8 up, 8 in 2024-04-14T15:19:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[37795]: osdmap e430: 8 total, 8 up, 8 in 2024-04-14T15:19:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:52.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[29624]: osdmap e430: 8 total, 8 up, 8 in 2024-04-14T15:19:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:52 smithi071 ceph-mon[33150]: osdmap e430: 8 total, 8 up, 8 in 2024-04-14T15:19:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:19:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[37795]: pgmap v638: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[37795]: osdmap e431: 8 total, 8 up, 8 in 2024-04-14T15:19:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[29624]: pgmap v638: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[29624]: osdmap e431: 8 total, 8 up, 8 in 2024-04-14T15:19:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:54.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:19:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:19:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:53 smithi071 ceph-mon[33150]: pgmap v638: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:19:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:19:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:53 smithi071 ceph-mon[33150]: osdmap e431: 8 total, 8 up, 8 in 2024-04-14T15:19:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:19:54.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:54 smithi045 ceph-mon[37795]: osdmap e432: 8 total, 8 up, 8 in 2024-04-14T15:19:54.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:54.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:54 smithi045 ceph-mon[29624]: osdmap e432: 8 total, 8 up, 8 in 2024-04-14T15:19:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:54 smithi071 ceph-mon[33150]: osdmap e432: 8 total, 8 up, 8 in 2024-04-14T15:19:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:19:55.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:55.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:55.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: osdmap e433: 8 total, 8 up, 8 in 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: 305.8 deep-scrub starts 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[37795]: 305.8 deep-scrub ok 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: osdmap e433: 8 total, 8 up, 8 in 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: 305.8 deep-scrub starts 2024-04-14T15:19:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:55 smithi045 ceph-mon[29624]: 305.8 deep-scrub ok 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: osdmap e433: 8 total, 8 up, 8 in 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.8"}]: dispatch 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: 305.8 deep-scrub starts 2024-04-14T15:19:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:55 smithi071 ceph-mon[33150]: 305.8 deep-scrub ok 2024-04-14T15:19:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:19:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:19:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:19:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:57 smithi045 ceph-mon[29624]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-04-14T15:19:57.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:57 smithi045 ceph-mon[37795]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-04-14T15:19:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:57 smithi071 ceph-mon[33150]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-04-14T15:19:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:58.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:59.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:19:59.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:19:59 smithi045 ceph-mon[37795]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:19:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:19:59 smithi045 ceph-mon[29624]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:20:00.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:19:59 smithi071 ceph-mon[33150]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:20:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[37795]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-04-14T15:20:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[37795]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-04-14T15:20:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[37795]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-04-14T15:20:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[37795]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-04-14T15:20:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[29624]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-04-14T15:20:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[29624]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-04-14T15:20:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[29624]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-04-14T15:20:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:00 smithi045 ceph-mon[29624]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-04-14T15:20:01.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:00 smithi071 ceph-mon[33150]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-04-14T15:20:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:00 smithi071 ceph-mon[33150]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-04-14T15:20:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:00 smithi071 ceph-mon[33150]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-04-14T15:20:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:00 smithi071 ceph-mon[33150]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-04-14T15:20:01.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:01 smithi045 ceph-mon[37795]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 911 B/s wr, 3 op/s 2024-04-14T15:20:01.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:01 smithi045 ceph-mon[29624]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 911 B/s wr, 3 op/s 2024-04-14T15:20:02.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:01 smithi071 ceph-mon[33150]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 911 B/s wr, 3 op/s 2024-04-14T15:20:03.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:03 smithi045 ceph-mon[29624]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 231 B/s wr, 1 op/s 2024-04-14T15:20:03.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:03 smithi045 ceph-mon[37795]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 231 B/s wr, 1 op/s 2024-04-14T15:20:04.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:03 smithi071 ceph-mon[33150]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 578 B/s rd, 231 B/s wr, 1 op/s 2024-04-14T15:20:04.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:04.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:04 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:04.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:04 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:04 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:05 smithi045 ceph-mon[37795]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-04-14T15:20:05.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:05 smithi045 ceph-mon[29624]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-04-14T15:20:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:05 smithi071 ceph-mon[33150]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-04-14T15:20:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:07 smithi045 ceph-mon[29624]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 173 B/s wr, 1 op/s 2024-04-14T15:20:07.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:07 smithi045 ceph-mon[37795]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 173 B/s wr, 1 op/s 2024-04-14T15:20:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:07 smithi071 ceph-mon[33150]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 173 B/s wr, 1 op/s 2024-04-14T15:20:09.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:09 smithi045 ceph-mon[37795]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-14T15:20:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:09 smithi045 ceph-mon[29624]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-14T15:20:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:09 smithi071 ceph-mon[33150]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-14T15:20:11.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:11 smithi045 ceph-mon[37795]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:11.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:11 smithi045 ceph-mon[29624]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:12.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:11 smithi071 ceph-mon[33150]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:13 smithi045 ceph-mon[29624]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:13 smithi045 ceph-mon[37795]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:13 smithi071 ceph-mon[33150]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:20:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:14.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:14.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:14.997 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:20:14 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:20:14.728+0000 7f4e6caa2700 -1 snap_mapper.add_oid found existing snaps mapped on 305:139cc77c:test-rados-api-smithi045-74757-80::foo:2, removing 2024-04-14T15:20:14.997 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:20:14 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:20:14.728+0000 7ff0a9ce3700 -1 snap_mapper.add_oid found existing snaps mapped on 305:139cc77c:test-rados-api-smithi045-74757-80::foo:2, removing 2024-04-14T15:20:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:15.107 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:20:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:20:14.727+0000 7f6aa019c700 -1 snap_mapper.add_oid found existing snaps mapped on 305:139cc77c:test-rados-api-smithi045-74757-80::foo:2, removing 2024-04-14T15:20:15.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[37795]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-04-14T15:20:15.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:15.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[29624]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:15.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:15 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:15 smithi071 ceph-mon[33150]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-04-14T15:20:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:15 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-79"}]: dispatch 2024-04-14T15:20:16.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:16 smithi045 ceph-mon[29624]: osdmap e434: 8 total, 8 up, 8 in 2024-04-14T15:20:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:16 smithi045 ceph-mon[37795]: osdmap e434: 8 total, 8 up, 8 in 2024-04-14T15:20:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:16 smithi071 ceph-mon[33150]: osdmap e434: 8 total, 8 up, 8 in 2024-04-14T15:20:18.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:17 smithi071 ceph-mon[33150]: pgmap v654: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-04-14T15:20:18.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:17 smithi071 ceph-mon[33150]: osdmap e435: 8 total, 8 up, 8 in 2024-04-14T15:20:18.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:18.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:17 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:18.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[29624]: pgmap v654: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-04-14T15:20:18.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[29624]: osdmap e435: 8 total, 8 up, 8 in 2024-04-14T15:20:18.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:18.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:18.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[37795]: pgmap v654: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-04-14T15:20:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[37795]: osdmap e435: 8 total, 8 up, 8 in 2024-04-14T15:20:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:17 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:19.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:19.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:18 smithi071 ceph-mon[33150]: osdmap e436: 8 total, 8 up, 8 in 2024-04-14T15:20:19.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:19.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:19.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:18 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[29624]: osdmap e436: 8 total, 8 up, 8 in 2024-04-14T15:20:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:19.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[37795]: osdmap e436: 8 total, 8 up, 8 in 2024-04-14T15:20:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:19.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:18 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 112 op/s 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: osdmap e437: 8 total, 8 up, 8 in 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:19 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:20:20.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 112 op/s 2024-04-14T15:20:20.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:20.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: osdmap e437: 8 total, 8 up, 8 in 2024-04-14T15:20:20.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 0 B/s wr, 112 op/s 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: osdmap e437: 8 total, 8 up, 8 in 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:20.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:19 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: osdmap e438: 8 total, 8 up, 8 in 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:20:21.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:20 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:20:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: osdmap e438: 8 total, 8 up, 8 in 2024-04-14T15:20:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: osdmap e438: 8 total, 8 up, 8 in 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:20:21.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:20 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: osdmap e439: 8 total, 8 up, 8 in 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:21 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:22.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: osdmap e439: 8 total, 8 up, 8 in 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: osdmap e439: 8 total, 8 up, 8 in 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:22.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:21 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:23.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:22 smithi071 ceph-mon[33150]: osdmap e440: 8 total, 8 up, 8 in 2024-04-14T15:20:23.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:23.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:22 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:23.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[29624]: osdmap e440: 8 total, 8 up, 8 in 2024-04-14T15:20:23.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:23.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:23.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[37795]: osdmap e440: 8 total, 8 up, 8 in 2024-04-14T15:20:23.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:23.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:22 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-04-14T15:20:24.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-mon[33150]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-04-14T15:20:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-mon[33150]: osdmap e441: 8 total, 8 up, 8 in 2024-04-14T15:20:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:23 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[29624]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-04-14T15:20:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[29624]: osdmap e441: 8 total, 8 up, 8 in 2024-04-14T15:20:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[37795]: pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-04-14T15:20:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[37795]: osdmap e441: 8 total, 8 up, 8 in 2024-04-14T15:20:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:23 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-04-14T15:20:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-04-14T15:20:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:24 smithi071 ceph-mon[33150]: osdmap e442: 8 total, 8 up, 8 in 2024-04-14T15:20:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:24 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:25.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-04-14T15:20:25.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[37795]: osdmap e442: 8 total, 8 up, 8 in 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[29624]: osdmap e442: 8 total, 8 up, 8 in 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:25.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: osdmap e443: 8 total, 8 up, 8 in 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:25 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: osdmap e443: 8 total, 8 up, 8 in 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: pgmap v666: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: osdmap e443: 8 total, 8 up, 8 in 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:26.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:25 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-81"}]: dispatch 2024-04-14T15:20:26.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:26 smithi045 ceph-mon[29624]: osdmap e444: 8 total, 8 up, 8 in 2024-04-14T15:20:26.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:26 smithi045 ceph-mon[37795]: osdmap e444: 8 total, 8 up, 8 in 2024-04-14T15:20:26.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:26 smithi071 ceph-mon[33150]: osdmap e444: 8 total, 8 up, 8 in 2024-04-14T15:20:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:27 smithi071 ceph-mon[33150]: pgmap v669: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-04-14T15:20:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:27 smithi071 ceph-mon[33150]: osdmap e445: 8 total, 8 up, 8 in 2024-04-14T15:20:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:27 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:28.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[37795]: pgmap v669: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-04-14T15:20:28.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[37795]: osdmap e445: 8 total, 8 up, 8 in 2024-04-14T15:20:28.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:28.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:28.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[29624]: pgmap v669: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-04-14T15:20:28.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[29624]: osdmap e445: 8 total, 8 up, 8 in 2024-04-14T15:20:28.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:28.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:27 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: osdmap e446: 8 total, 8 up, 8 in 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:29.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:29.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: osdmap e446: 8 total, 8 up, 8 in 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: osdmap e446: 8 total, 8 up, 8 in 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:29.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:29 smithi071 ceph-mon[33150]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-04-14T15:20:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:29 smithi071 ceph-mon[33150]: osdmap e447: 8 total, 8 up, 8 in 2024-04-14T15:20:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:29 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:30.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[37795]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-04-14T15:20:30.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:30.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[37795]: osdmap e447: 8 total, 8 up, 8 in 2024-04-14T15:20:30.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[29624]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[29624]: osdmap e447: 8 total, 8 up, 8 in 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:30.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:29 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:30 smithi071 ceph-mon[33150]: osdmap e448: 8 total, 8 up, 8 in 2024-04-14T15:20:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:31.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:31.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[37795]: osdmap e448: 8 total, 8 up, 8 in 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[29624]: osdmap e448: 8 total, 8 up, 8 in 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:31.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:31 smithi071 ceph-mon[33150]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-04-14T15:20:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:31 smithi071 ceph-mon[33150]: osdmap e449: 8 total, 8 up, 8 in 2024-04-14T15:20:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:31 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:32.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[29624]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-04-14T15:20:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[29624]: osdmap e449: 8 total, 8 up, 8 in 2024-04-14T15:20:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:32.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[37795]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-04-14T15:20:32.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:32.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[37795]: osdmap e449: 8 total, 8 up, 8 in 2024-04-14T15:20:32.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:32.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:31 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:32 smithi071 ceph-mon[33150]: osdmap e450: 8 total, 8 up, 8 in 2024-04-14T15:20:33.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:33.148 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:32 smithi045 ceph-mon[29624]: osdmap e450: 8 total, 8 up, 8 in 2024-04-14T15:20:33.148 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:33.148 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:32 smithi045 ceph-mon[37795]: osdmap e450: 8 total, 8 up, 8 in 2024-04-14T15:20:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:33 smithi071 ceph-mon[33150]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:33 smithi071 ceph-mon[33150]: osdmap e451: 8 total, 8 up, 8 in 2024-04-14T15:20:34.107 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:34.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:33 smithi045 ceph-mon[29624]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:34.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:33 smithi045 ceph-mon[29624]: osdmap e451: 8 total, 8 up, 8 in 2024-04-14T15:20:34.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:33 smithi045 ceph-mon[37795]: pgmap v678: 228 pgs: 228 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:34.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:33 smithi045 ceph-mon[37795]: osdmap e451: 8 total, 8 up, 8 in 2024-04-14T15:20:35.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: osdmap e452: 8 total, 8 up, 8 in 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: osdmap e452: 8 total, 8 up, 8 in 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: osdmap e452: 8 total, 8 up, 8 in 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-83"}]: dispatch 2024-04-14T15:20:35.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:36.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:35 smithi045 ceph-mon[29624]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:20:36.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:35 smithi045 ceph-mon[29624]: osdmap e453: 8 total, 8 up, 8 in 2024-04-14T15:20:36.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:35 smithi045 ceph-mon[37795]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:20:36.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:35 smithi045 ceph-mon[37795]: osdmap e453: 8 total, 8 up, 8 in 2024-04-14T15:20:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:35 smithi071 ceph-mon[33150]: pgmap v681: 228 pgs: 228 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-04-14T15:20:36.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:35 smithi071 ceph-mon[33150]: osdmap e453: 8 total, 8 up, 8 in 2024-04-14T15:20:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[29624]: osdmap e454: 8 total, 8 up, 8 in 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[37795]: osdmap e454: 8 total, 8 up, 8 in 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:36 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:36 smithi071 ceph-mon[33150]: osdmap e454: 8 total, 8 up, 8 in 2024-04-14T15:20:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:37.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:36 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:38.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: pgmap v684: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: osdmap e455: 8 total, 8 up, 8 in 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: pgmap v684: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:38.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:38.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: osdmap e455: 8 total, 8 up, 8 in 2024-04-14T15:20:38.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:38.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:38.248 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:37 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: pgmap v684: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: osdmap e455: 8 total, 8 up, 8 in 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:38.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:37 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[37795]: osdmap e456: 8 total, 8 up, 8 in 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[29624]: osdmap e456: 8 total, 8 up, 8 in 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:38 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:38 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:38 smithi071 ceph-mon[33150]: osdmap e456: 8 total, 8 up, 8 in 2024-04-14T15:20:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:39.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:38 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[37795]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-14T15:20:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[37795]: osdmap e457: 8 total, 8 up, 8 in 2024-04-14T15:20:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[29624]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-14T15:20:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[29624]: osdmap e457: 8 total, 8 up, 8 in 2024-04-14T15:20:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:39 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:39 smithi071 ceph-mon[33150]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-14T15:20:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_tier","val": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:20:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:39 smithi071 ceph-mon[33150]: osdmap e457: 8 total, 8 up, 8 in 2024-04-14T15:20:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:40.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:39 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:41.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:41.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[29624]: osdmap e458: 8 total, 8 up, 8 in 2024-04-14T15:20:41.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:41.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[37795]: osdmap e458: 8 total, 8 up, 8 in 2024-04-14T15:20:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:40 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:40 smithi071 ceph-mon[33150]: osdmap e458: 8 total, 8 up, 8 in 2024-04-14T15:20:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:41.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:40 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[37795]: pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-14T15:20:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[37795]: osdmap e459: 8 total, 8 up, 8 in 2024-04-14T15:20:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[29624]: pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-14T15:20:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:42.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:41 smithi045 ceph-mon[29624]: osdmap e459: 8 total, 8 up, 8 in 2024-04-14T15:20:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:41 smithi071 ceph-mon[33150]: pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-04-14T15:20:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:41 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:42.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:41 smithi071 ceph-mon[33150]: osdmap e459: 8 total, 8 up, 8 in 2024-04-14T15:20:43.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:42 smithi045 ceph-mon[37795]: osdmap e460: 8 total, 8 up, 8 in 2024-04-14T15:20:43.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:42 smithi045 ceph-mon[29624]: osdmap e460: 8 total, 8 up, 8 in 2024-04-14T15:20:43.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:42 smithi071 ceph-mon[33150]: osdmap e460: 8 total, 8 up, 8 in 2024-04-14T15:20:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: osdmap e461: 8 total, 8 up, 8 in 2024-04-14T15:20:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: osdmap e461: 8 total, 8 up, 8 in 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: pgmap v693: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: osdmap e461: 8 total, 8 up, 8 in 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-85"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:44.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:45.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:44 smithi045 ceph-mon[29624]: osdmap e462: 8 total, 8 up, 8 in 2024-04-14T15:20:45.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:45.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:44 smithi045 ceph-mon[37795]: osdmap e462: 8 total, 8 up, 8 in 2024-04-14T15:20:45.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:44 smithi071 ceph-mon[33150]: osdmap e462: 8 total, 8 up, 8 in 2024-04-14T15:20:45.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18074 ms) 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22230 ms) 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3012 ms) 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3017 ms) 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3017 ms) 2024-04-14T15:20:45.949 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3020 ms) 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4031 ms) 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38470 ms) 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17063 ms) 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4023 ms) 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-04-14T15:20:45.950 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3027 ms) 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25070 ms) 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14285 ms) 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (6047 ms) 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4020 ms) 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-04-14T15:20:45.951 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: waiting for scrubs... 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: done waiting 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25077 ms) 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10071 ms) 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9065 ms) 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9051 ms) 2024-04-14T15:20:45.952 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-04-14T15:20:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[29624]: pgmap v696: 196 pgs: 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[29624]: osdmap e463: 8 total, 8 up, 8 in 2024-04-14T15:20:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[37795]: pgmap v696: 196 pgs: 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[37795]: osdmap e463: 8 total, 8 up, 8 in 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:45 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:45 smithi071 ceph-mon[33150]: pgmap v696: 196 pgs: 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:45 smithi071 ceph-mon[33150]: osdmap e463: 8 total, 8 up, 8 in 2024-04-14T15:20:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:45 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:45 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[29624]: osdmap e464: 8 total, 8 up, 8 in 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[37795]: osdmap e464: 8 total, 8 up, 8 in 2024-04-14T15:20:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:46 smithi071 ceph-mon[33150]: osdmap e464: 8 total, 8 up, 8 in 2024-04-14T15:20:47.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[37795]: pgmap v699: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[37795]: osdmap e465: 8 total, 8 up, 8 in 2024-04-14T15:20:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:48.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[29624]: pgmap v699: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:48.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[29624]: osdmap e465: 8 total, 8 up, 8 in 2024-04-14T15:20:48.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:48.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:47 smithi071 ceph-mon[33150]: pgmap v699: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-14T15:20:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:47 smithi071 ceph-mon[33150]: osdmap e465: 8 total, 8 up, 8 in 2024-04-14T15:20:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: osdmap e466: 8 total, 8 up, 8 in 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]': finished 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[29624]: osdmap e467: 8 total, 8 up, 8 in 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: osdmap e466: 8 total, 8 up, 8 in 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]': finished 2024-04-14T15:20:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:48 smithi045 ceph-mon[37795]: osdmap e467: 8 total, 8 up, 8 in 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: osdmap e466: 8 total, 8 up, 8 in 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]: dispatch 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_tier","val": "test-rados-api-smithi045-74757-89-test-flush"}]': finished 2024-04-14T15:20:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:48 smithi071 ceph-mon[33150]: osdmap e467: 8 total, 8 up, 8 in 2024-04-14T15:20:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[37795]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[37795]: osdmap e468: 8 total, 8 up, 8 in 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[29624]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:49 smithi045 ceph-mon[29624]: osdmap e468: 8 total, 8 up, 8 in 2024-04-14T15:20:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:49 smithi071 ceph-mon[33150]: pgmap v702: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:20:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:20:50.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:49 smithi071 ceph-mon[33150]: osdmap e468: 8 total, 8 up, 8 in 2024-04-14T15:20:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:51.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:50 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:51.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:20:51.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:50 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[29624]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[29624]: osdmap e469: 8 total, 8 up, 8 in 2024-04-14T15:20:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[37795]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:51 smithi045 ceph-mon[37795]: osdmap e469: 8 total, 8 up, 8 in 2024-04-14T15:20:52.258 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:51 smithi071 ceph-mon[33150]: pgmap v705: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:20:52.258 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:20:52.258 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:51 smithi071 ceph-mon[33150]: osdmap e469: 8 total, 8 up, 8 in 2024-04-14T15:20:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[29624]: osdmap e470: 8 total, 8 up, 8 in 2024-04-14T15:20:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[37795]: osdmap e470: 8 total, 8 up, 8 in 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:52 smithi071 ceph-mon[33150]: osdmap e470: 8 total, 8 up, 8 in 2024-04-14T15:20:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:53.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-87"}]: dispatch 2024-04-14T15:20:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:53 smithi045 ceph-mon[29624]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:53 smithi045 ceph-mon[29624]: osdmap e471: 8 total, 8 up, 8 in 2024-04-14T15:20:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:53 smithi045 ceph-mon[37795]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:53 smithi045 ceph-mon[37795]: osdmap e471: 8 total, 8 up, 8 in 2024-04-14T15:20:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:53 smithi071 ceph-mon[33150]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:20:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:53 smithi071 ceph-mon[33150]: osdmap e471: 8 total, 8 up, 8 in 2024-04-14T15:20:54.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:20:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:20:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[37795]: osdmap e472: 8 total, 8 up, 8 in 2024-04-14T15:20:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[29624]: osdmap e472: 8 total, 8 up, 8 in 2024-04-14T15:20:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:20:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:54 smithi071 ceph-mon[33150]: osdmap e472: 8 total, 8 up, 8 in 2024-04-14T15:20:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:55.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:20:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: osdmap e473: 8 total, 8 up, 8 in 2024-04-14T15:20:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: osdmap e473: 8 total, 8 up, 8 in 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: osdmap e473: 8 total, 8 up, 8 in 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:20:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:20:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:20:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:20:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:57 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:57.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:57 smithi071 ceph-mon[33150]: osdmap e474: 8 total, 8 up, 8 in 2024-04-14T15:20:57.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:57 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:57.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:57 smithi045 ceph-mon[29624]: osdmap e474: 8 total, 8 up, 8 in 2024-04-14T15:20:57.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:57 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:20:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:57 smithi045 ceph-mon[37795]: osdmap e474: 8 total, 8 up, 8 in 2024-04-14T15:20:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:58 smithi071 ceph-mon[33150]: pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:58 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:58 smithi071 ceph-mon[33150]: osdmap e475: 8 total, 8 up, 8 in 2024-04-14T15:20:58.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[37795]: pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[37795]: osdmap e475: 8 total, 8 up, 8 in 2024-04-14T15:20:58.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[29624]: pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:20:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:20:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[29624]: osdmap e475: 8 total, 8 up, 8 in 2024-04-14T15:20:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:20:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:59 smithi071 ceph-mon[33150]: osdmap e476: 8 total, 8 up, 8 in 2024-04-14T15:20:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:20:59.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:20:59 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:20:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[37795]: osdmap e476: 8 total, 8 up, 8 in 2024-04-14T15:20:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:20:59.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:20:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[29624]: osdmap e476: 8 total, 8 up, 8 in 2024-04-14T15:20:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:20:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:20:59.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:20:59 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-90"}]: dispatch 2024-04-14T15:21:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:00 smithi071 ceph-mon[33150]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-14T15:21:00.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:00 smithi071 ceph-mon[33150]: osdmap e477: 8 total, 8 up, 8 in 2024-04-14T15:21:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:00 smithi045 ceph-mon[37795]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-14T15:21:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:00 smithi045 ceph-mon[37795]: osdmap e477: 8 total, 8 up, 8 in 2024-04-14T15:21:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:00 smithi045 ceph-mon[29624]: pgmap v717: 228 pgs: 228 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-04-14T15:21:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:00 smithi045 ceph-mon[29624]: osdmap e477: 8 total, 8 up, 8 in 2024-04-14T15:21:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:01 smithi071 ceph-mon[33150]: osdmap e478: 8 total, 8 up, 8 in 2024-04-14T15:21:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:01.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[37795]: osdmap e478: 8 total, 8 up, 8 in 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[29624]: osdmap e478: 8 total, 8 up, 8 in 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:01.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:02 smithi071 ceph-mon[33150]: pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:21:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:02 smithi071 ceph-mon[33150]: osdmap e479: 8 total, 8 up, 8 in 2024-04-14T15:21:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:02.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:02 smithi071 ceph-mon[33150]: osdmap e480: 8 total, 8 up, 8 in 2024-04-14T15:21:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[37795]: pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:21:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[37795]: osdmap e479: 8 total, 8 up, 8 in 2024-04-14T15:21:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[37795]: osdmap e480: 8 total, 8 up, 8 in 2024-04-14T15:21:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[29624]: pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-14T15:21:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[29624]: osdmap e479: 8 total, 8 up, 8 in 2024-04-14T15:21:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:02 smithi045 ceph-mon[29624]: osdmap e480: 8 total, 8 up, 8 in 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: osdmap e481: 8 total, 8 up, 8 in 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:03.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: osdmap e481: 8 total, 8 up, 8 in 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: osdmap e481: 8 total, 8 up, 8 in 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:03.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:03.399 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:03 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-92"}]: dispatch 2024-04-14T15:21:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:04 smithi071 ceph-mon[33150]: pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:04.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:04 smithi045 ceph-mon[37795]: pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:04 smithi045 ceph-mon[29624]: pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:05 smithi045 ceph-mon[29624]: osdmap e482: 8 total, 8 up, 8 in 2024-04-14T15:21:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:05 smithi045 ceph-mon[37795]: osdmap e482: 8 total, 8 up, 8 in 2024-04-14T15:21:05.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:05 smithi071 ceph-mon[33150]: osdmap e482: 8 total, 8 up, 8 in 2024-04-14T15:21:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[29624]: pgmap v726: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[29624]: osdmap e483: 8 total, 8 up, 8 in 2024-04-14T15:21:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[37795]: pgmap v726: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[37795]: osdmap e483: 8 total, 8 up, 8 in 2024-04-14T15:21:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:06 smithi071 ceph-mon[33150]: pgmap v726: 196 pgs: 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:06 smithi071 ceph-mon[33150]: osdmap e483: 8 total, 8 up, 8 in 2024-04-14T15:21:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[29624]: osdmap e484: 8 total, 8 up, 8 in 2024-04-14T15:21:07.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[37795]: osdmap e484: 8 total, 8 up, 8 in 2024-04-14T15:21:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:07 smithi071 ceph-mon[33150]: osdmap e484: 8 total, 8 up, 8 in 2024-04-14T15:21:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:08 smithi045 ceph-mon[29624]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:08 smithi045 ceph-mon[29624]: osdmap e485: 8 total, 8 up, 8 in 2024-04-14T15:21:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:08 smithi045 ceph-mon[37795]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:08 smithi045 ceph-mon[37795]: osdmap e485: 8 total, 8 up, 8 in 2024-04-14T15:21:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:08 smithi071 ceph-mon[33150]: pgmap v729: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-04-14T15:21:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:08 smithi071 ceph-mon[33150]: osdmap e485: 8 total, 8 up, 8 in 2024-04-14T15:21:09.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:09 smithi045 ceph-mon[37795]: osdmap e486: 8 total, 8 up, 8 in 2024-04-14T15:21:09.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:09 smithi045 ceph-mon[29624]: osdmap e486: 8 total, 8 up, 8 in 2024-04-14T15:21:09.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:09 smithi071 ceph-mon[33150]: osdmap e486: 8 total, 8 up, 8 in 2024-04-14T15:21:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:10 smithi045 ceph-mon[37795]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-14T15:21:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:10 smithi045 ceph-mon[29624]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-14T15:21:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:10 smithi071 ceph-mon[33150]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-14T15:21:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:12 smithi045 ceph-mon[37795]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:21:12.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:12 smithi045 ceph-mon[29624]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:21:12.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:12 smithi071 ceph-mon[33150]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-14T15:21:13.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:14 smithi071 ceph-mon[33150]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:21:14.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:13 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:14 smithi045 ceph-mon[29624]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:21:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:14 smithi045 ceph-mon[37795]: pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-14T15:21:15.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:16 smithi045 ceph-mon[29624]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-14T15:21:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:16 smithi045 ceph-mon[37795]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-14T15:21:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:16 smithi071 ceph-mon[33150]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-14T15:21:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:18 smithi045 ceph-mon[29624]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 870 B/s wr, 3 op/s 2024-04-14T15:21:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:18 smithi045 ceph-mon[37795]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 870 B/s wr, 3 op/s 2024-04-14T15:21:18.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:18 smithi071 ceph-mon[33150]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 870 B/s wr, 3 op/s 2024-04-14T15:21:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[37795]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 786 B/s wr, 3 op/s 2024-04-14T15:21:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[37795]: osdmap e487: 8 total, 8 up, 8 in 2024-04-14T15:21:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:21:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[29624]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 786 B/s wr, 3 op/s 2024-04-14T15:21:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[29624]: osdmap e487: 8 total, 8 up, 8 in 2024-04-14T15:21:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:20 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:21:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:20 smithi071 ceph-mon[33150]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 786 B/s wr, 3 op/s 2024-04-14T15:21:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:20 smithi071 ceph-mon[33150]: osdmap e487: 8 total, 8 up, 8 in 2024-04-14T15:21:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:20 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:21:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:21:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:21:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:21:21.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:21:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[37795]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi071 to 3248M 2024-04-14T15:21:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[37795]: Adjusting osd_memory_target on smithi045 to 3249M 2024-04-14T15:21:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[29624]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi071 to 3248M 2024-04-14T15:21:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:22 smithi045 ceph-mon[29624]: Adjusting osd_memory_target on smithi045 to 3249M 2024-04-14T15:21:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:22 smithi071 ceph-mon[33150]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:22 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi071 to 3248M 2024-04-14T15:21:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:22 smithi071 ceph-mon[33150]: Adjusting osd_memory_target on smithi045 to 3249M 2024-04-14T15:21:24.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:24 smithi071 ceph-mon[33150]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:24.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:24 smithi045 ceph-mon[37795]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:24 smithi045 ceph-mon[29624]: pgmap v740: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:25.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:26 smithi045 ceph-mon[37795]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:26 smithi045 ceph-mon[29624]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:26 smithi071 ceph-mon[33150]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:28 smithi045 ceph-mon[29624]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:28 smithi045 ceph-mon[37795]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:28.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:28.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:28 smithi071 ceph-mon[33150]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[29624]: osdmap e488: 8 total, 8 up, 8 in 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:30 smithi045 ceph-mon[37795]: osdmap e488: 8 total, 8 up, 8 in 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-94"}]: dispatch 2024-04-14T15:21:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:30 smithi071 ceph-mon[33150]: osdmap e488: 8 total, 8 up, 8 in 2024-04-14T15:21:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:32 smithi071 ceph-mon[33150]: pgmap v745: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-04-14T15:21:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:32 smithi071 ceph-mon[33150]: osdmap e489: 8 total, 8 up, 8 in 2024-04-14T15:21:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:32.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:32 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[37795]: pgmap v745: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-04-14T15:21:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[37795]: osdmap e489: 8 total, 8 up, 8 in 2024-04-14T15:21:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[29624]: pgmap v745: 196 pgs: 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 614 B/s rd, 0 op/s 2024-04-14T15:21:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[29624]: osdmap e489: 8 total, 8 up, 8 in 2024-04-14T15:21:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:32 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:33 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:33 smithi071 ceph-mon[33150]: osdmap e490: 8 total, 8 up, 8 in 2024-04-14T15:21:33.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:33.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:33.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[29624]: osdmap e490: 8 total, 8 up, 8 in 2024-04-14T15:21:33.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:33.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:33.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[37795]: osdmap e490: 8 total, 8 up, 8 in 2024-04-14T15:21:33.398 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:21:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:34 smithi071 ceph-mon[33150]: pgmap v748: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:34 smithi071 ceph-mon[33150]: osdmap e491: 8 total, 8 up, 8 in 2024-04-14T15:21:34.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:34 smithi045 ceph-mon[29624]: pgmap v748: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:34 smithi045 ceph-mon[29624]: osdmap e491: 8 total, 8 up, 8 in 2024-04-14T15:21:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:34 smithi045 ceph-mon[37795]: pgmap v748: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:34.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:34 smithi045 ceph-mon[37795]: osdmap e491: 8 total, 8 up, 8 in 2024-04-14T15:21:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:36.107 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:21:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=cleanup t=2024-04-14T15:21:35.82526532Z level=info msg="Completed cleanup jobs" duration=3.25891ms 2024-04-14T15:21:36.107 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:21:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore.transactions t=2024-04-14T15:21:35.832694686Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-04-14T15:21:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:36 smithi045 ceph-mon[29624]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:21:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:36 smithi045 ceph-mon[37795]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:21:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:36 smithi071 ceph-mon[33150]: pgmap v750: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-04-14T15:21:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:38.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:38 smithi045 ceph-mon[37795]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:21:38.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:38 smithi045 ceph-mon[29624]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:21:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:38 smithi071 ceph-mon[33150]: pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:21:40.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:40 smithi045 ceph-mon[29624]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 823 B/s wr, 2 op/s 2024-04-14T15:21:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:40 smithi045 ceph-mon[37795]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 823 B/s wr, 2 op/s 2024-04-14T15:21:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:40 smithi071 ceph-mon[33150]: pgmap v752: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 823 B/s wr, 2 op/s 2024-04-14T15:21:42.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:42 smithi045 ceph-mon[29624]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 726 B/s wr, 2 op/s 2024-04-14T15:21:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:42 smithi045 ceph-mon[37795]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 726 B/s wr, 2 op/s 2024-04-14T15:21:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:42 smithi071 ceph-mon[33150]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 726 B/s wr, 2 op/s 2024-04-14T15:21:43.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:43.497 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:21:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:21:43.124+0000 7fb7c7837700 -1 snap_mapper.add_oid found existing snaps mapped on 109:d5f2d88e:test-rados-api-smithi045-74757-97::foo:21, removing 2024-04-14T15:21:43.497 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:21:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:21:43.124+0000 7ff0ac4e8700 -1 snap_mapper.add_oid found existing snaps mapped on 109:d5f2d88e:test-rados-api-smithi045-74757-97::foo:21, removing 2024-04-14T15:21:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:43.607 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:21:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:21:43.124+0000 7f84c797f700 -1 snap_mapper.add_oid found existing snaps mapped on 109:d5f2d88e:test-rados-api-smithi045-74757-97::foo:21, removing 2024-04-14T15:21:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:44 smithi071 ceph-mon[33150]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-04-14T15:21:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:44 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:44.358 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[29624]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-04-14T15:21:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[37795]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:44.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:44 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-96"}]: dispatch 2024-04-14T15:21:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:45 smithi045 ceph-mon[29624]: osdmap e492: 8 total, 8 up, 8 in 2024-04-14T15:21:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:45 smithi045 ceph-mon[37795]: osdmap e492: 8 total, 8 up, 8 in 2024-04-14T15:21:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:45 smithi071 ceph-mon[33150]: osdmap e492: 8 total, 8 up, 8 in 2024-04-14T15:21:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[29624]: pgmap v756: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-04-14T15:21:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[29624]: osdmap e493: 8 total, 8 up, 8 in 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[37795]: pgmap v756: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[37795]: osdmap e493: 8 total, 8 up, 8 in 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:46 smithi071 ceph-mon[33150]: pgmap v756: 196 pgs: 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-04-14T15:21:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:46 smithi071 ceph-mon[33150]: osdmap e493: 8 total, 8 up, 8 in 2024-04-14T15:21:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:46 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[29624]: osdmap e494: 8 total, 8 up, 8 in 2024-04-14T15:21:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:47.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[37795]: osdmap e494: 8 total, 8 up, 8 in 2024-04-14T15:21:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:47.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:47 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:47.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:47 smithi071 ceph-mon[33150]: osdmap e494: 8 total, 8 up, 8 in 2024-04-14T15:21:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:47 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[29624]: pgmap v759: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[29624]: osdmap e495: 8 total, 8 up, 8 in 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[37795]: pgmap v759: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[37795]: osdmap e495: 8 total, 8 up, 8 in 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:48 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:48 smithi071 ceph-mon[33150]: pgmap v759: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:21:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:21:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:48 smithi071 ceph-mon[33150]: osdmap e495: 8 total, 8 up, 8 in 2024-04-14T15:21:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:48 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]: dispatch 2024-04-14T15:21:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:21:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]': finished 2024-04-14T15:21:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: osdmap e496: 8 total, 8 up, 8 in 2024-04-14T15:21:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[29624]: osdmap e497: 8 total, 8 up, 8 in 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]': finished 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: osdmap e496: 8 total, 8 up, 8 in 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:49 smithi045 ceph-mon[37795]: osdmap e497: 8 total, 8 up, 8 in 2024-04-14T15:21:49.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-98", "mode": "writeback"}]': finished 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: osdmap e496: 8 total, 8 up, 8 in 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:49 smithi071 ceph-mon[33150]: osdmap e497: 8 total, 8 up, 8 in 2024-04-14T15:21:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[29624]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[37795]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:50 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:50 smithi071 ceph-mon[33150]: pgmap v762: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:50 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[29624]: osdmap e498: 8 total, 8 up, 8 in 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:21:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:21:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[37795]: osdmap e498: 8 total, 8 up, 8 in 2024-04-14T15:21:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:51.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:51 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:51 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:21:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:21:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:51 smithi071 ceph-mon[33150]: osdmap e498: 8 total, 8 up, 8 in 2024-04-14T15:21:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:51 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]: dispatch 2024-04-14T15:21:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[29624]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]': finished 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[29624]: osdmap e499: 8 total, 8 up, 8 in 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[37795]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]': finished 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[37795]: osdmap e499: 8 total, 8 up, 8 in 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:52 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:52 smithi071 ceph-mon[33150]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:21:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_count","val": "1"}]': finished 2024-04-14T15:21:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:52 smithi071 ceph-mon[33150]: osdmap e499: 8 total, 8 up, 8 in 2024-04-14T15:21:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:52 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:21:53.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: osdmap e500: 8 total, 8 up, 8 in 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]': finished 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[29624]: osdmap e501: 8 total, 8 up, 8 in 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: osdmap e500: 8 total, 8 up, 8 in 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]': finished 2024-04-14T15:21:53.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:53 smithi045 ceph-mon[37795]: osdmap e501: 8 total, 8 up, 8 in 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: osdmap e500: 8 total, 8 up, 8 in 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]: dispatch 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-98","var": "target_max_objects","val": "250"}]': finished 2024-04-14T15:21:53.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-mon[33150]: osdmap e501: 8 total, 8 up, 8 in 2024-04-14T15:21:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:54 smithi071 ceph-mon[33150]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:54 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:54.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:21:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:21:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[37795]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:54.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[29624]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:21:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:54.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:54 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:21:55.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[37795]: osdmap e502: 8 total, 8 up, 8 in 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[29624]: osdmap e502: 8 total, 8 up, 8 in 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:55 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:21:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:21:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:55 smithi071 ceph-mon[33150]: osdmap e502: 8 total, 8 up, 8 in 2024-04-14T15:21:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:55 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]: dispatch 2024-04-14T15:21:56.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[37795]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[37795]: osdmap e503: 8 total, 8 up, 8 in 2024-04-14T15:21:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[29624]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-mon[29624]: osdmap e503: 8 total, 8 up, 8 in 2024-04-14T15:21:56.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:56 smithi071 ceph-mon[33150]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:21:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:56 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-98"}]': finished 2024-04-14T15:21:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:56 smithi071 ceph-mon[33150]: osdmap e503: 8 total, 8 up, 8 in 2024-04-14T15:21:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:21:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:21:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:21:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[37795]: osdmap e504: 8 total, 8 up, 8 in 2024-04-14T15:21:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[37795]: osdmap e505: 8 total, 8 up, 8 in 2024-04-14T15:21:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:57.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[29624]: osdmap e504: 8 total, 8 up, 8 in 2024-04-14T15:21:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[29624]: osdmap e505: 8 total, 8 up, 8 in 2024-04-14T15:21:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:57.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:57 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:57.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:57 smithi071 ceph-mon[33150]: osdmap e504: 8 total, 8 up, 8 in 2024-04-14T15:21:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:57 smithi071 ceph-mon[33150]: osdmap e505: 8 total, 8 up, 8 in 2024-04-14T15:21:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:57 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:21:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: pgmap v774: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:21:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: osdmap e506: 8 total, 8 up, 8 in 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: pgmap v774: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: osdmap e506: 8 total, 8 up, 8 in 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:21:58 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: pgmap v774: 196 pgs: 196 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: osdmap e506: 8 total, 8 up, 8 in 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:21:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:21:58 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[37795]: pgmap v777: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[37795]: osdmap e507: 8 total, 8 up, 8 in 2024-04-14T15:22:00.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[29624]: pgmap v777: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[29624]: osdmap e507: 8 total, 8 up, 8 in 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:00 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:00 smithi071 ceph-mon[33150]: pgmap v777: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:00 smithi071 ceph-mon[33150]: osdmap e507: 8 total, 8 up, 8 in 2024-04-14T15:22:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:00 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: osdmap e508: 8 total, 8 up, 8 in 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]': finished 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: osdmap e508: 8 total, 8 up, 8 in 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:01.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:01 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]': finished 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-6", "overlaypool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: osdmap e508: 8 total, 8 up, 8 in 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]: dispatch 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:01 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-100", "mode": "writeback"}]': finished 2024-04-14T15:22:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: osdmap e509: 8 total, 8 up, 8 in 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: osdmap e510: 8 total, 8 up, 8 in 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: osdmap e509: 8 total, 8 up, 8 in 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:02.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: osdmap e510: 8 total, 8 up, 8 in 2024-04-14T15:22:02.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:02.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:02 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: osdmap e509: 8 total, 8 up, 8 in 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: osdmap e510: 8 total, 8 up, 8 in 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:02.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:02 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:04.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:03 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:04 smithi071 ceph-mon[33150]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:04 smithi071 ceph-mon[33150]: osdmap e511: 8 total, 8 up, 8 in 2024-04-14T15:22:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:04.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:04 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[29624]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[29624]: osdmap e511: 8 total, 8 up, 8 in 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[37795]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[37795]: osdmap e511: 8 total, 8 up, 8 in 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:04 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: osdmap e512: 8 total, 8 up, 8 in 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[29624]: osdmap e513: 8 total, 8 up, 8 in 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: osdmap e512: 8 total, 8 up, 8 in 2024-04-14T15:22:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-04-14T15:22:05.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:05 smithi045 ceph-mon[37795]: osdmap e513: 8 total, 8 up, 8 in 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: osdmap e512: 8 total, 8 up, 8 in 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-04-14T15:22:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:05 smithi071 ceph-mon[33150]: osdmap e513: 8 total, 8 up, 8 in 2024-04-14T15:22:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:06 smithi071 ceph-mon[33150]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:06 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[37795]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[29624]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:22:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:07 smithi071 ceph-mon[33150]: osdmap e514: 8 total, 8 up, 8 in 2024-04-14T15:22:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:07 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:22:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[37795]: osdmap e514: 8 total, 8 up, 8 in 2024-04-14T15:22:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]': finished 2024-04-14T15:22:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[29624]: osdmap e514: 8 total, 8 up, 8 in 2024-04-14T15:22:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:07.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:07 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]: dispatch 2024-04-14T15:22:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:08 smithi071 ceph-mon[33150]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:08 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:08 smithi071 ceph-mon[33150]: osdmap e515: 8 total, 8 up, 8 in 2024-04-14T15:22:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[29624]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[29624]: osdmap e515: 8 total, 8 up, 8 in 2024-04-14T15:22:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[37795]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-100"}]': finished 2024-04-14T15:22:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:08 smithi045 ceph-mon[37795]: osdmap e515: 8 total, 8 up, 8 in 2024-04-14T15:22:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:09 smithi071 ceph-mon[33150]: osdmap e516: 8 total, 8 up, 8 in 2024-04-14T15:22:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:09 smithi045 ceph-mon[37795]: osdmap e516: 8 total, 8 up, 8 in 2024-04-14T15:22:09.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:09 smithi045 ceph-mon[29624]: osdmap e516: 8 total, 8 up, 8 in 2024-04-14T15:22:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:10 smithi071 ceph-mon[33150]: pgmap v792: 196 pgs: 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:10 smithi071 ceph-mon[33150]: osdmap e517: 8 total, 8 up, 8 in 2024-04-14T15:22:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:10 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[29624]: pgmap v792: 196 pgs: 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[29624]: osdmap e517: 8 total, 8 up, 8 in 2024-04-14T15:22:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[37795]: pgmap v792: 196 pgs: 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[37795]: osdmap e517: 8 total, 8 up, 8 in 2024-04-14T15:22:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:10 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:11 smithi071 ceph-mon[33150]: osdmap e518: 8 total, 8 up, 8 in 2024-04-14T15:22:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:22:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:11 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[37795]: osdmap e518: 8 total, 8 up, 8 in 2024-04-14T15:22:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:22:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[29624]: osdmap e518: 8 total, 8 up, 8 in 2024-04-14T15:22:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-04-14T15:22:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:11 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-04-14T15:22:12.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:12 smithi071 ceph-mon[33150]: pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:12.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:22:12.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:12 smithi071 ceph-mon[33150]: osdmap e519: 8 total, 8 up, 8 in 2024-04-14T15:22:12.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:12.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:12 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[29624]: pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:22:12.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[29624]: osdmap e519: 8 total, 8 up, 8 in 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[37795]: pgmap v795: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[37795]: osdmap e519: 8 total, 8 up, 8 in 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:12 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: osdmap e520: 8 total, 8 up, 8 in 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:22:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:13 smithi071 ceph-mon[33150]: osdmap e521: 8 total, 8 up, 8 in 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: osdmap e520: 8 total, 8 up, 8 in 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:13.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[37795]: osdmap e521: 8 total, 8 up, 8 in 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: osdmap e520: 8 total, 8 up, 8 in 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-04-14T15:22:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:13 smithi045 ceph-mon[29624]: osdmap e521: 8 total, 8 up, 8 in 2024-04-14T15:22:14.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-mon[33150]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:14 smithi071 ceph-mon[33150]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[29624]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[29624]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[37795]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-6"}]: dispatch 2024-04-14T15:22:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3319280306' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:14 smithi045 ceph-mon[37795]: from='client.35519 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-6", "tierpool": "test-rados-api-smithi045-74757-102"}]: dispatch 2024-04-14T15:22:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:15 smithi071 ceph-mon[33150]: osdmap e522: 8 total, 8 up, 8 in 2024-04-14T15:22:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:15 smithi045 ceph-mon[37795]: osdmap e522: 8 total, 8 up, 8 in 2024-04-14T15:22:15.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:15.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:15 smithi045 ceph-mon[29624]: osdmap e522: 8 total, 8 up, 8 in 2024-04-14T15:22:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:16 smithi071 ceph-mon[33150]: pgmap v801: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:16 smithi071 ceph-mon[33150]: osdmap e523: 8 total, 8 up, 8 in 2024-04-14T15:22:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[37795]: pgmap v801: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[37795]: osdmap e523: 8 total, 8 up, 8 in 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[29624]: pgmap v801: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[29624]: osdmap e523: 8 total, 8 up, 8 in 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:17 smithi071 ceph-mon[33150]: osdmap e524: 8 total, 8 up, 8 in 2024-04-14T15:22:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:17 smithi071 ceph-mon[33150]: pgmap v804: 164 pgs: 164 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[37795]: osdmap e524: 8 total, 8 up, 8 in 2024-04-14T15:22:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[37795]: pgmap v804: 164 pgs: 164 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[29624]: osdmap e524: 8 total, 8 up, 8 in 2024-04-14T15:22:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:17 smithi045 ceph-mon[29624]: pgmap v804: 164 pgs: 164 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:18 smithi071 ceph-mon[33150]: osdmap e525: 8 total, 8 up, 8 in 2024-04-14T15:22:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:18 smithi045 ceph-mon[37795]: osdmap e525: 8 total, 8 up, 8 in 2024-04-14T15:22:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:18 smithi045 ceph-mon[29624]: osdmap e525: 8 total, 8 up, 8 in 2024-04-14T15:22:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:19 smithi071 ceph-mon[33150]: osdmap e526: 8 total, 8 up, 8 in 2024-04-14T15:22:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:19 smithi071 ceph-mon[33150]: pgmap v807: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[37795]: osdmap e526: 8 total, 8 up, 8 in 2024-04-14T15:22:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[37795]: pgmap v807: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:19.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi045-74757-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[29624]: osdmap e526: 8 total, 8 up, 8 in 2024-04-14T15:22:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:19 smithi045 ceph-mon[29624]: pgmap v807: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:20 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:20 smithi071 ceph-mon[33150]: osdmap e527: 8 total, 8 up, 8 in 2024-04-14T15:22:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:20 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:20 smithi045 ceph-mon[37795]: osdmap e527: 8 total, 8 up, 8 in 2024-04-14T15:22:20.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:20 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:20.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:20 smithi045 ceph-mon[29624]: osdmap e527: 8 total, 8 up, 8 in 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: osdmap e528: 8 total, 8 up, 8 in 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: pgmap v810: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:22:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: osdmap e528: 8 total, 8 up, 8 in 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: pgmap v810: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:22:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: osdmap e528: 8 total, 8 up, 8 in 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: pgmap v810: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:22:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:22 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:22 smithi045 ceph-mon[29624]: osdmap e529: 8 total, 8 up, 8 in 2024-04-14T15:22:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:22 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:22 smithi045 ceph-mon[37795]: osdmap e529: 8 total, 8 up, 8 in 2024-04-14T15:22:22.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:22 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:22.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:22 smithi071 ceph-mon[33150]: osdmap e529: 8 total, 8 up, 8 in 2024-04-14T15:22:23.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[29624]: osdmap e530: 8 total, 8 up, 8 in 2024-04-14T15:22:23.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[29624]: pgmap v813: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[37795]: osdmap e530: 8 total, 8 up, 8 in 2024-04-14T15:22:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:23 smithi045 ceph-mon[37795]: pgmap v813: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:23.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:23 smithi071 ceph-mon[33150]: osdmap e530: 8 total, 8 up, 8 in 2024-04-14T15:22:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:23 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:23 smithi071 ceph-mon[33150]: pgmap v813: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:24.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[29624]: osdmap e531: 8 total, 8 up, 8 in 2024-04-14T15:22:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[37795]: osdmap e531: 8 total, 8 up, 8 in 2024-04-14T15:22:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:24.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-mon[33150]: osdmap e531: 8 total, 8 up, 8 in 2024-04-14T15:22:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: osdmap e532: 8 total, 8 up, 8 in 2024-04-14T15:22:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: pgmap v816: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: osdmap e532: 8 total, 8 up, 8 in 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: pgmap v816: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:25 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: osdmap e532: 8 total, 8 up, 8 in 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: pgmap v816: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:25 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[37795]: osdmap e533: 8 total, 8 up, 8 in 2024-04-14T15:22:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[29624]: osdmap e533: 8 total, 8 up, 8 in 2024-04-14T15:22:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:26 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-107", "overlaypool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:26.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:26 smithi071 ceph-mon[33150]: osdmap e533: 8 total, 8 up, 8 in 2024-04-14T15:22:26.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:26 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: osdmap e534: 8 total, 8 up, 8 in 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[29624]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: osdmap e534: 8 total, 8 up, 8 in 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:27 smithi045 ceph-mon[37795]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:27.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-107-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: osdmap e534: 8 total, 8 up, 8 in 2024-04-14T15:22:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]: dispatch 2024-04-14T15:22:27.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:27 smithi071 ceph-mon[33150]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:22:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]': finished 2024-04-14T15:22:28.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: osdmap e535: 8 total, 8 up, 8 in 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]': finished 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: osdmap e535: 8 total, 8 up, 8 in 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:28.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-107"}]': finished 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/399718899' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: osdmap e535: 8 total, 8 up, 8 in 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]: dispatch 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:28.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:29.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:29.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[37795]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:29.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[37795]: osdmap e536: 8 total, 8 up, 8 in 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[37795]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[37795]: osdmap e537: 8 total, 8 up, 8 in 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[29624]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[29624]: osdmap e536: 8 total, 8 up, 8 in 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[29624]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:22:29.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:29 smithi045 ceph-mon[29624]: osdmap e537: 8 total, 8 up, 8 in 2024-04-14T15:22:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:29 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:29 smithi071 ceph-mon[33150]: from='client.48277 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-107", "tierpool": "test-rados-api-smithi045-74757-107-cache"}]': finished 2024-04-14T15:22:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:29 smithi071 ceph-mon[33150]: osdmap e536: 8 total, 8 up, 8 in 2024-04-14T15:22:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:29 smithi071 ceph-mon[33150]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:22:29.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:29 smithi071 ceph-mon[33150]: osdmap e537: 8 total, 8 up, 8 in 2024-04-14T15:22:30.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:30 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:30.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:30 smithi045 ceph-mon[37795]: osdmap e538: 8 total, 8 up, 8 in 2024-04-14T15:22:30.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:30 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:30.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:30 smithi045 ceph-mon[29624]: osdmap e538: 8 total, 8 up, 8 in 2024-04-14T15:22:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:30 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:30 smithi071 ceph-mon[33150]: osdmap e538: 8 total, 8 up, 8 in 2024-04-14T15:22:31.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[37795]: pgmap v825: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:31.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:31 smithi045 ceph-mon[29624]: pgmap v825: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:31 smithi071 ceph-mon[33150]: pgmap v825: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:32.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:32.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[37795]: osdmap e539: 8 total, 8 up, 8 in 2024-04-14T15:22:32.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[29624]: osdmap e539: 8 total, 8 up, 8 in 2024-04-14T15:22:32.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:32 smithi071 ceph-mon[33150]: osdmap e539: 8 total, 8 up, 8 in 2024-04-14T15:22:32.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:33.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[37795]: osdmap e540: 8 total, 8 up, 8 in 2024-04-14T15:22:33.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[37795]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:33.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:33.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[37795]: osdmap e541: 8 total, 8 up, 8 in 2024-04-14T15:22:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[29624]: osdmap e540: 8 total, 8 up, 8 in 2024-04-14T15:22:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[29624]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:33.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:33 smithi045 ceph-mon[29624]: osdmap e541: 8 total, 8 up, 8 in 2024-04-14T15:22:33.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:33 smithi071 ceph-mon[33150]: osdmap e540: 8 total, 8 up, 8 in 2024-04-14T15:22:33.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:33 smithi071 ceph-mon[33150]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:33.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:33.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:33 smithi071 ceph-mon[33150]: osdmap e541: 8 total, 8 up, 8 in 2024-04-14T15:22:34.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:34.455 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:34.455 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[29624]: osdmap e542: 8 total, 8 up, 8 in 2024-04-14T15:22:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:34.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:34.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[37795]: osdmap e542: 8 total, 8 up, 8 in 2024-04-14T15:22:34.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:34.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:34.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:34 smithi071 ceph-mon[33150]: osdmap e542: 8 total, 8 up, 8 in 2024-04-14T15:22:34.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[37795]: pgmap v831: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[37795]: osdmap e543: 8 total, 8 up, 8 in 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[29624]: pgmap v831: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[29624]: osdmap e543: 8 total, 8 up, 8 in 2024-04-14T15:22:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:35 smithi071 ceph-mon[33150]: pgmap v831: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:35 smithi071 ceph-mon[33150]: osdmap e543: 8 total, 8 up, 8 in 2024-04-14T15:22:35.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[29624]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[29624]: osdmap e544: 8 total, 8 up, 8 in 2024-04-14T15:22:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[37795]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[37795]: osdmap e544: 8 total, 8 up, 8 in 2024-04-14T15:22:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:36.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:36 smithi071 ceph-mon[33150]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:36 smithi071 ceph-mon[33150]: osdmap e544: 8 total, 8 up, 8 in 2024-04-14T15:22:36.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:37.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[37795]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:37.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:37.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[37795]: osdmap e545: 8 total, 8 up, 8 in 2024-04-14T15:22:37.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:37.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[29624]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:37.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:37.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[29624]: osdmap e545: 8 total, 8 up, 8 in 2024-04-14T15:22:37.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:37 smithi071 ceph-mon[33150]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-109", "overlaypool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:37 smithi071 ceph-mon[33150]: osdmap e545: 8 total, 8 up, 8 in 2024-04-14T15:22:37.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]: dispatch 2024-04-14T15:22:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[37795]: osdmap e546: 8 total, 8 up, 8 in 2024-04-14T15:22:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[29624]: osdmap e546: 8 total, 8 up, 8 in 2024-04-14T15:22:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:38 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:22:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-109-cache", "mode": "writeback"}]': finished 2024-04-14T15:22:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:38 smithi071 ceph-mon[33150]: osdmap e546: 8 total, 8 up, 8 in 2024-04-14T15:22:38.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:22:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[37795]: pgmap v837: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[37795]: osdmap e547: 8 total, 8 up, 8 in 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[29624]: pgmap v837: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[29624]: osdmap e547: 8 total, 8 up, 8 in 2024-04-14T15:22:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:39.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:39 smithi071 ceph-mon[33150]: pgmap v837: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:39.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:22:39.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:39 smithi071 ceph-mon[33150]: osdmap e547: 8 total, 8 up, 8 in 2024-04-14T15:22:39.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:22:41.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:41.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[29624]: osdmap e548: 8 total, 8 up, 8 in 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[29624]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[37795]: osdmap e548: 8 total, 8 up, 8 in 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:41.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:41 smithi045 ceph-mon[37795]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:22:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:41 smithi071 ceph-mon[33150]: osdmap e548: 8 total, 8 up, 8 in 2024-04-14T15:22:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:22:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:41 smithi071 ceph-mon[33150]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[37795]: osdmap e549: 8 total, 8 up, 8 in 2024-04-14T15:22:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-04-14T15:22:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[29624]: osdmap e549: 8 total, 8 up, 8 in 2024-04-14T15:22:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-04-14T15:22:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:42 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:22:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:22:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:42 smithi071 ceph-mon[33150]: osdmap e549: 8 total, 8 up, 8 in 2024-04-14T15:22:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-04-14T15:22:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-04-14T15:22:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: osdmap e550: 8 total, 8 up, 8 in 2024-04-14T15:22:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: osdmap e550: 8 total, 8 up, 8 in 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: osdmap e550: 8 total, 8 up, 8 in 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:22:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:44.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[29624]: osdmap e551: 8 total, 8 up, 8 in 2024-04-14T15:22:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[37795]: osdmap e551: 8 total, 8 up, 8 in 2024-04-14T15:22:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:44 smithi071 ceph-mon[33150]: osdmap e551: 8 total, 8 up, 8 in 2024-04-14T15:22:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]: dispatch 2024-04-14T15:22:44.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[29624]: osdmap e552: 8 total, 8 up, 8 in 2024-04-14T15:22:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[29624]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[37795]: osdmap e552: 8 total, 8 up, 8 in 2024-04-14T15:22:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[37795]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:45 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-109", "tierpool": "test-rados-api-smithi045-74757-109-cache"}]': finished 2024-04-14T15:22:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:45 smithi071 ceph-mon[33150]: osdmap e552: 8 total, 8 up, 8 in 2024-04-14T15:22:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:45 smithi071 ceph-mon[33150]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:22:45.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:45 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:46.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:46 smithi045 ceph-mon[29624]: osdmap e553: 8 total, 8 up, 8 in 2024-04-14T15:22:46.746 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:46.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:46 smithi045 ceph-mon[37795]: osdmap e553: 8 total, 8 up, 8 in 2024-04-14T15:22:46.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:46 smithi071 ceph-mon[33150]: osdmap e553: 8 total, 8 up, 8 in 2024-04-14T15:22:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[37795]: osdmap e554: 8 total, 8 up, 8 in 2024-04-14T15:22:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[37795]: pgmap v849: 172 pgs: 172 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[29624]: osdmap e554: 8 total, 8 up, 8 in 2024-04-14T15:22:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:47 smithi045 ceph-mon[29624]: pgmap v849: 172 pgs: 172 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:47.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:47 smithi071 ceph-mon[33150]: osdmap e554: 8 total, 8 up, 8 in 2024-04-14T15:22:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:47 smithi071 ceph-mon[33150]: pgmap v849: 172 pgs: 172 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:48 smithi071 ceph-mon[33150]: osdmap e555: 8 total, 8 up, 8 in 2024-04-14T15:22:48.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:48.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:48.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[29624]: osdmap e555: 8 total, 8 up, 8 in 2024-04-14T15:22:48.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[37795]: osdmap e555: 8 total, 8 up, 8 in 2024-04-14T15:22:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]: dispatch 2024-04-14T15:22:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:49 smithi071 ceph-mon[33150]: osdmap e556: 8 total, 8 up, 8 in 2024-04-14T15:22:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:49 smithi071 ceph-mon[33150]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[29624]: osdmap e556: 8 total, 8 up, 8 in 2024-04-14T15:22:49.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[29624]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1696950588' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-109"}]': finished 2024-04-14T15:22:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[37795]: osdmap e556: 8 total, 8 up, 8 in 2024-04-14T15:22:49.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:49 smithi045 ceph-mon[37795]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:50 smithi071 ceph-mon[33150]: osdmap e557: 8 total, 8 up, 8 in 2024-04-14T15:22:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:50 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:50.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[29624]: osdmap e557: 8 total, 8 up, 8 in 2024-04-14T15:22:50.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:50.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:50.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[37795]: osdmap e557: 8 total, 8 up, 8 in 2024-04-14T15:22:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:50.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:50 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:51 smithi071 ceph-mon[33150]: osdmap e558: 8 total, 8 up, 8 in 2024-04-14T15:22:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:51 smithi071 ceph-mon[33150]: pgmap v855: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[29624]: osdmap e558: 8 total, 8 up, 8 in 2024-04-14T15:22:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:51.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[29624]: pgmap v855: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[37795]: osdmap e558: 8 total, 8 up, 8 in 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]: dispatch 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[37795]: pgmap v855: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-04-14T15:22:51.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/792645919' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi045-74757-104"}]': finished 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: osdmap e559: 8 total, 8 up, 8 in 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:52.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: osdmap e559: 8 total, 8 up, 8 in 2024-04-14T15:22:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: osdmap e559: 8 total, 8 up, 8 in 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:52.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-04-14T15:22:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:53 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:53 smithi071 ceph-mon[33150]: osdmap e560: 8 total, 8 up, 8 in 2024-04-14T15:22:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:53 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:53 smithi071 ceph-mon[33150]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[37795]: osdmap e560: 8 total, 8 up, 8 in 2024-04-14T15:22:53.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[37795]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi045-74757-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[29624]: osdmap e560: 8 total, 8 up, 8 in 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:53.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:53 smithi045 ceph-mon[29624]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:22:54.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:22:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:22:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:54 smithi071 ceph-mon[33150]: osdmap e561: 8 total, 8 up, 8 in 2024-04-14T15:22:54.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:54.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:54 smithi045 ceph-mon[37795]: osdmap e561: 8 total, 8 up, 8 in 2024-04-14T15:22:54.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:54 smithi045 ceph-mon[29624]: osdmap e561: 8 total, 8 up, 8 in 2024-04-14T15:22:54.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:22:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:22:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:55 smithi071 ceph-mon[33150]: osdmap e562: 8 total, 8 up, 8 in 2024-04-14T15:22:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:55 smithi071 ceph-mon[33150]: pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:55 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:55.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:22:55.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[29624]: osdmap e562: 8 total, 8 up, 8 in 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[29624]: pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi045-74757-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[37795]: osdmap e562: 8 total, 8 up, 8 in 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[37795]: pgmap v861: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:55.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:55 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:22:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:56 smithi071 ceph-mon[33150]: osdmap e563: 8 total, 8 up, 8 in 2024-04-14T15:22:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[29624]: osdmap e563: 8 total, 8 up, 8 in 2024-04-14T15:22:56.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[37795]: osdmap e563: 8 total, 8 up, 8 in 2024-04-14T15:22:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:22:56.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:22:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:22:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:22:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:57 smithi071 ceph-mon[33150]: osdmap e564: 8 total, 8 up, 8 in 2024-04-14T15:22:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:57 smithi071 ceph-mon[33150]: pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[29624]: osdmap e564: 8 total, 8 up, 8 in 2024-04-14T15:22:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[29624]: pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:22:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[37795]: osdmap e564: 8 total, 8 up, 8 in 2024-04-14T15:22:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:22:57.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:57 smithi045 ceph-mon[37795]: pgmap v864: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:22:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:58 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:58 smithi071 ceph-mon[33150]: osdmap e565: 8 total, 8 up, 8 in 2024-04-14T15:22:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:58 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[29624]: osdmap e565: 8 total, 8 up, 8 in 2024-04-14T15:22:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[37795]: osdmap e565: 8 total, 8 up, 8 in 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:22:58.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:22:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:22:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:59 smithi071 ceph-mon[33150]: osdmap e566: 8 total, 8 up, 8 in 2024-04-14T15:22:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:59 smithi071 ceph-mon[33150]: pgmap v867: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:22:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:22:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:22:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[29624]: osdmap e566: 8 total, 8 up, 8 in 2024-04-14T15:22:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[29624]: pgmap v867: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:22:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:59.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:22:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[37795]: osdmap e566: 8 total, 8 up, 8 in 2024-04-14T15:22:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[37795]: pgmap v867: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:22:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:22:59.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:22:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:00 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:00 smithi071 ceph-mon[33150]: osdmap e567: 8 total, 8 up, 8 in 2024-04-14T15:23:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:00 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:00 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:00 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:23:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[37795]: osdmap e567: 8 total, 8 up, 8 in 2024-04-14T15:23:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[29624]: osdmap e567: 8 total, 8 up, 8 in 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]: dispatch 2024-04-14T15:23:00.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:00 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:23:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:01 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:23:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:01 smithi071 ceph-mon[33150]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:01.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:01 smithi071 ceph-mon[33150]: osdmap e568: 8 total, 8 up, 8 in 2024-04-14T15:23:01.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:23:01.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[29624]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:01.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[29624]: osdmap e568: 8 total, 8 up, 8 in 2024-04-14T15:23:01.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-112"}]': finished 2024-04-14T15:23:01.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[37795]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:01.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:01 smithi045 ceph-mon[37795]: osdmap e568: 8 total, 8 up, 8 in 2024-04-14T15:23:02.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:02 smithi071 ceph-mon[33150]: osdmap e569: 8 total, 8 up, 8 in 2024-04-14T15:23:02.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:02 smithi045 ceph-mon[29624]: osdmap e569: 8 total, 8 up, 8 in 2024-04-14T15:23:02.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:02 smithi045 ceph-mon[37795]: osdmap e569: 8 total, 8 up, 8 in 2024-04-14T15:23:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[37795]: pgmap v872: 172 pgs: 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[37795]: osdmap e570: 8 total, 8 up, 8 in 2024-04-14T15:23:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:03.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[29624]: pgmap v872: 172 pgs: 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[29624]: osdmap e570: 8 total, 8 up, 8 in 2024-04-14T15:23:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:03.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:03 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:04.048 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:03 smithi071 ceph-mon[33150]: pgmap v872: 172 pgs: 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:04.048 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:03 smithi071 ceph-mon[33150]: osdmap e570: 8 total, 8 up, 8 in 2024-04-14T15:23:04.048 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:04.048 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:03 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:04.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:04.897 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: osdmap e571: 8 total, 8 up, 8 in 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: osdmap e572: 8 total, 8 up, 8 in 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: osdmap e571: 8 total, 8 up, 8 in 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:04.898 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:04.899 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:04.899 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: osdmap e572: 8 total, 8 up, 8 in 2024-04-14T15:23:04.899 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:04.899 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: osdmap e571: 8 total, 8 up, 8 in 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: osdmap e572: 8 total, 8 up, 8 in 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:05.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[37795]: pgmap v875: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[37795]: osdmap e573: 8 total, 8 up, 8 in 2024-04-14T15:23:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:05.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[29624]: pgmap v875: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[29624]: osdmap e573: 8 total, 8 up, 8 in 2024-04-14T15:23:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:05.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:05 smithi071 ceph-mon[33150]: pgmap v875: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:05 smithi071 ceph-mon[33150]: osdmap e573: 8 total, 8 up, 8 in 2024-04-14T15:23:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]: dispatch 2024-04-14T15:23:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]': finished 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[37795]: osdmap e574: 8 total, 8 up, 8 in 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]': finished 2024-04-14T15:23:06.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:06 smithi045 ceph-mon[29624]: osdmap e574: 8 total, 8 up, 8 in 2024-04-14T15:23:07.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:06 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:07.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:06 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-114", "mode": "writeback"}]': finished 2024-04-14T15:23:07.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:06 smithi071 ceph-mon[33150]: osdmap e574: 8 total, 8 up, 8 in 2024-04-14T15:23:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[29624]: pgmap v878: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:07.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:07.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[37795]: pgmap v878: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:07.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:07.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:07 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:07 smithi071 ceph-mon[33150]: pgmap v878: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-14T15:23:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:07 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:08.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:08.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[29624]: osdmap e575: 8 total, 8 up, 8 in 2024-04-14T15:23:08.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:08.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:08.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:08.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[37795]: osdmap e575: 8 total, 8 up, 8 in 2024-04-14T15:23:08.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:08.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:08 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:09.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:08 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:09.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:08 smithi071 ceph-mon[33150]: osdmap e575: 8 total, 8 up, 8 in 2024-04-14T15:23:09.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:09.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:08 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]: dispatch 2024-04-14T15:23:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[29624]: pgmap v881: 204 pgs: 204 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:23:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:09.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[29624]: osdmap e576: 8 total, 8 up, 8 in 2024-04-14T15:23:09.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[37795]: pgmap v881: 204 pgs: 204 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:23:09.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:09.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:09.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:09 smithi045 ceph-mon[37795]: osdmap e576: 8 total, 8 up, 8 in 2024-04-14T15:23:10.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:09 smithi071 ceph-mon[33150]: pgmap v881: 204 pgs: 204 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-14T15:23:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:09 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:09 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-114"}]': finished 2024-04-14T15:23:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:09 smithi071 ceph-mon[33150]: osdmap e576: 8 total, 8 up, 8 in 2024-04-14T15:23:10.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:10 smithi045 ceph-mon[29624]: osdmap e577: 8 total, 8 up, 8 in 2024-04-14T15:23:10.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:10 smithi045 ceph-mon[37795]: osdmap e577: 8 total, 8 up, 8 in 2024-04-14T15:23:11.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:10 smithi071 ceph-mon[33150]: osdmap e577: 8 total, 8 up, 8 in 2024-04-14T15:23:11.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[29624]: pgmap v884: 172 pgs: 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:11.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[29624]: osdmap e578: 8 total, 8 up, 8 in 2024-04-14T15:23:11.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:11.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:11.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[37795]: pgmap v884: 172 pgs: 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:11.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[37795]: osdmap e578: 8 total, 8 up, 8 in 2024-04-14T15:23:11.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:11.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:11 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:12.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:11 smithi071 ceph-mon[33150]: pgmap v884: 172 pgs: 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:12.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:11 smithi071 ceph-mon[33150]: osdmap e578: 8 total, 8 up, 8 in 2024-04-14T15:23:12.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:12.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:11 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:12.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:12 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:12.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:12 smithi045 ceph-mon[29624]: osdmap e579: 8 total, 8 up, 8 in 2024-04-14T15:23:12.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:12 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:12.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:12 smithi045 ceph-mon[37795]: osdmap e579: 8 total, 8 up, 8 in 2024-04-14T15:23:13.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:12 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:13.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:12 smithi071 ceph-mon[33150]: osdmap e579: 8 total, 8 up, 8 in 2024-04-14T15:23:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[29624]: pgmap v887: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[29624]: osdmap e580: 8 total, 8 up, 8 in 2024-04-14T15:23:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:13.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[37795]: pgmap v887: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[37795]: osdmap e580: 8 total, 8 up, 8 in 2024-04-14T15:23:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:13.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:14.058 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:13 smithi071 ceph-mon[33150]: pgmap v887: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:14.058 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:13 smithi071 ceph-mon[33150]: osdmap e580: 8 total, 8 up, 8 in 2024-04-14T15:23:14.058 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:14.058 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:14.058 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:14.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:14.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:14.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: osdmap e581: 8 total, 8 up, 8 in 2024-04-14T15:23:14.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: osdmap e582: 8 total, 8 up, 8 in 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: osdmap e581: 8 total, 8 up, 8 in 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:14.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: osdmap e582: 8 total, 8 up, 8 in 2024-04-14T15:23:14.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: osdmap e581: 8 total, 8 up, 8 in 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: osdmap e582: 8 total, 8 up, 8 in 2024-04-14T15:23:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[29624]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]': finished 2024-04-14T15:23:15.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[29624]: osdmap e583: 8 total, 8 up, 8 in 2024-04-14T15:23:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[37795]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]': finished 2024-04-14T15:23:15.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:15 smithi045 ceph-mon[37795]: osdmap e583: 8 total, 8 up, 8 in 2024-04-14T15:23:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:15 smithi071 ceph-mon[33150]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]: dispatch 2024-04-14T15:23:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:15 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-116", "mode": "writeback"}]': finished 2024-04-14T15:23:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:15 smithi071 ceph-mon[33150]: osdmap e583: 8 total, 8 up, 8 in 2024-04-14T15:23:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[37795]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[37795]: 326.3 scrub starts 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[37795]: 326.3 scrub ok 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[29624]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[29624]: 326.3 scrub starts 2024-04-14T15:23:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:16 smithi045 ceph-mon[29624]: 326.3 scrub ok 2024-04-14T15:23:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:16 smithi071 ceph-mon[33150]: from='mon.? v1:172.21.15.45:0/1178444869' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.3"}]: dispatch 2024-04-14T15:23:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:16 smithi071 ceph-mon[33150]: 326.3 scrub starts 2024-04-14T15:23:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:16 smithi071 ceph-mon[33150]: 326.3 scrub ok 2024-04-14T15:23:17.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:17 smithi045 ceph-mon[37795]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:17.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:17 smithi045 ceph-mon[29624]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:18.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:17 smithi071 ceph-mon[33150]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:23:19.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:19 smithi045 ceph-mon[37795]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:23:19.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:19 smithi045 ceph-mon[29624]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:23:20.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:19 smithi071 ceph-mon[33150]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-14T15:23:21.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:21 smithi045 ceph-mon[29624]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:23:21.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:21 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:23:21.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:21 smithi045 ceph-mon[37795]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:23:21.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:21 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:23:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:21 smithi071 ceph-mon[33150]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:23:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:21 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:23:23.986 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:23 smithi071 ceph-mon[33150]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-04-14T15:23:23.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:23 smithi045 ceph-mon[37795]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-04-14T15:23:23.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:23 smithi045 ceph-mon[29624]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-04-14T15:23:24.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:24.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:24.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:24.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:25.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[29624]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:25.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:25.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:25.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[37795]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:25.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:25.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:25 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:25 smithi071 ceph-mon[33150]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:25 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:25 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:26.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:23:26.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:26 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:26 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:26 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:26 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:23:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:26 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:23:27.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:26 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:27 smithi071 ceph-mon[33150]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:23:28.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:27 smithi045 ceph-mon[37795]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:23:28.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:27 smithi045 ceph-mon[29624]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:23:29.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:29.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:29.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:29.499 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:30 smithi045 ceph-mon[37795]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:23:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:30 smithi045 ceph-mon[29624]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:23:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:30 smithi071 ceph-mon[33150]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-14T15:23:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[37795]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:23:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[37795]: osdmap e584: 8 total, 8 up, 8 in 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[29624]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[29624]: osdmap e584: 8 total, 8 up, 8 in 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:32 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:32 smithi071 ceph-mon[33150]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:23:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:32 smithi071 ceph-mon[33150]: osdmap e584: 8 total, 8 up, 8 in 2024-04-14T15:23:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:32 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:33.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:33.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: osdmap e585: 8 total, 8 up, 8 in 2024-04-14T15:23:33.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: osdmap e585: 8 total, 8 up, 8 in 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:33.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:33 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: osdmap e585: 8 total, 8 up, 8 in 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]: dispatch 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:33 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-116"}]': finished 2024-04-14T15:23:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:34 smithi071 ceph-mon[33150]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:23:34.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:34 smithi071 ceph-mon[33150]: osdmap e586: 8 total, 8 up, 8 in 2024-04-14T15:23:34.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:34 smithi045 ceph-mon[37795]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:23:34.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:34 smithi045 ceph-mon[37795]: osdmap e586: 8 total, 8 up, 8 in 2024-04-14T15:23:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:34 smithi045 ceph-mon[29624]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:23:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:34 smithi045 ceph-mon[29624]: osdmap e586: 8 total, 8 up, 8 in 2024-04-14T15:23:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:35.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:35 smithi045 ceph-mon[29624]: osdmap e587: 8 total, 8 up, 8 in 2024-04-14T15:23:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:35.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:35 smithi045 ceph-mon[37795]: osdmap e587: 8 total, 8 up, 8 in 2024-04-14T15:23:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:35 smithi071 ceph-mon[33150]: osdmap e587: 8 total, 8 up, 8 in 2024-04-14T15:23:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[29624]: pgmap v906: 172 pgs: 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[29624]: osdmap e588: 8 total, 8 up, 8 in 2024-04-14T15:23:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[37795]: pgmap v906: 172 pgs: 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[37795]: osdmap e588: 8 total, 8 up, 8 in 2024-04-14T15:23:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:36 smithi071 ceph-mon[33150]: pgmap v906: 172 pgs: 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:36 smithi071 ceph-mon[33150]: osdmap e588: 8 total, 8 up, 8 in 2024-04-14T15:23:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:36 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:37.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:37.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:37 smithi045 ceph-mon[29624]: osdmap e589: 8 total, 8 up, 8 in 2024-04-14T15:23:37.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:37.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:37 smithi045 ceph-mon[37795]: osdmap e589: 8 total, 8 up, 8 in 2024-04-14T15:23:37.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:37 smithi071 ceph-mon[33150]: osdmap e589: 8 total, 8 up, 8 in 2024-04-14T15:23:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[29624]: pgmap v909: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[29624]: osdmap e590: 8 total, 8 up, 8 in 2024-04-14T15:23:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:38.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[37795]: pgmap v909: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[37795]: osdmap e590: 8 total, 8 up, 8 in 2024-04-14T15:23:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:38.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:38 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:38 smithi071 ceph-mon[33150]: pgmap v909: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 636 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-04-14T15:23:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:38 smithi071 ceph-mon[33150]: osdmap e590: 8 total, 8 up, 8 in 2024-04-14T15:23:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:38 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:39.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[37795]: osdmap e591: 8 total, 8 up, 8 in 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[29624]: osdmap e591: 8 total, 8 up, 8 in 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:39.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:39 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:39.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:39 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:39 smithi071 ceph-mon[33150]: osdmap e591: 8 total, 8 up, 8 in 2024-04-14T15:23:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:39 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:39 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:40.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: osdmap e592: 8 total, 8 up, 8 in 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]': finished 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[37795]: osdmap e593: 8 total, 8 up, 8 in 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: osdmap e592: 8 total, 8 up, 8 in 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:40.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]': finished 2024-04-14T15:23:40.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:40 smithi045 ceph-mon[29624]: osdmap e593: 8 total, 8 up, 8 in 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: osdmap e592: 8 total, 8 up, 8 in 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]: dispatch 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-118", "mode": "writeback"}]': finished 2024-04-14T15:23:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:40 smithi071 ceph-mon[33150]: osdmap e593: 8 total, 8 up, 8 in 2024-04-14T15:23:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[37795]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[37795]: osdmap e594: 8 total, 8 up, 8 in 2024-04-14T15:23:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[29624]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[29624]: osdmap e594: 8 total, 8 up, 8 in 2024-04-14T15:23:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:42.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:42 smithi071 ceph-mon[33150]: pgmap v915: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:23:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:42 smithi071 ceph-mon[33150]: osdmap e594: 8 total, 8 up, 8 in 2024-04-14T15:23:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:43.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: osdmap e595: 8 total, 8 up, 8 in 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[37795]: osdmap e596: 8 total, 8 up, 8 in 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: osdmap e595: 8 total, 8 up, 8 in 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:43.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:43 smithi045 ceph-mon[29624]: osdmap e596: 8 total, 8 up, 8 in 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: osdmap e595: 8 total, 8 up, 8 in 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]: dispatch 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-118"}]': finished 2024-04-14T15:23:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:43 smithi071 ceph-mon[33150]: osdmap e596: 8 total, 8 up, 8 in 2024-04-14T15:23:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:44 smithi071 ceph-mon[33150]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:44.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:44.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[29624]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[37795]: pgmap v918: 204 pgs: 204 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:44.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:45.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:45 smithi045 ceph-mon[29624]: osdmap e597: 8 total, 8 up, 8 in 2024-04-14T15:23:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:45.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:45 smithi045 ceph-mon[37795]: osdmap e597: 8 total, 8 up, 8 in 2024-04-14T15:23:45.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:45.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:45 smithi071 ceph-mon[33150]: osdmap e597: 8 total, 8 up, 8 in 2024-04-14T15:23:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[29624]: pgmap v921: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:46.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[29624]: osdmap e598: 8 total, 8 up, 8 in 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[37795]: pgmap v921: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[37795]: osdmap e598: 8 total, 8 up, 8 in 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:46 smithi071 ceph-mon[33150]: pgmap v921: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:46 smithi071 ceph-mon[33150]: osdmap e598: 8 total, 8 up, 8 in 2024-04-14T15:23:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[29624]: osdmap e599: 8 total, 8 up, 8 in 2024-04-14T15:23:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:47.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[37795]: osdmap e599: 8 total, 8 up, 8 in 2024-04-14T15:23:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:47.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:47 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:47.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:47 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:47 smithi071 ceph-mon[33150]: osdmap e599: 8 total, 8 up, 8 in 2024-04-14T15:23:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:47 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[29624]: pgmap v924: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[29624]: osdmap e600: 8 total, 8 up, 8 in 2024-04-14T15:23:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:48.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[37795]: pgmap v924: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[37795]: osdmap e600: 8 total, 8 up, 8 in 2024-04-14T15:23:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:48.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:48 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:48 smithi071 ceph-mon[33150]: pgmap v924: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:23:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:48 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:48 smithi071 ceph-mon[33150]: osdmap e600: 8 total, 8 up, 8 in 2024-04-14T15:23:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:48 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[29624]: osdmap e601: 8 total, 8 up, 8 in 2024-04-14T15:23:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:49.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[37795]: osdmap e601: 8 total, 8 up, 8 in 2024-04-14T15:23:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:49.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:49.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:49 smithi071 ceph-mon[33150]: osdmap e601: 8 total, 8 up, 8 in 2024-04-14T15:23:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]: dispatch 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: pgmap v927: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]': finished 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: osdmap e602: 8 total, 8 up, 8 in 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: osdmap e603: 8 total, 8 up, 8 in 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: pgmap v927: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]': finished 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: osdmap e602: 8 total, 8 up, 8 in 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: osdmap e603: 8 total, 8 up, 8 in 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:50.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: pgmap v927: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-120", "mode": "writeback"}]': finished 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: osdmap e602: 8 total, 8 up, 8 in 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: osdmap e603: 8 total, 8 up, 8 in 2024-04-14T15:23:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:50.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]: dispatch 2024-04-14T15:23:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:51.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[29624]: osdmap e604: 8 total, 8 up, 8 in 2024-04-14T15:23:51.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:51.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:51 smithi045 ceph-mon[37795]: osdmap e604: 8 total, 8 up, 8 in 2024-04-14T15:23:51.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:51 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:23:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:51 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-120"}]': finished 2024-04-14T15:23:51.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:51 smithi071 ceph-mon[33150]: osdmap e604: 8 total, 8 up, 8 in 2024-04-14T15:23:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:52 smithi045 ceph-mon[29624]: pgmap v930: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:52.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:52 smithi045 ceph-mon[29624]: osdmap e605: 8 total, 8 up, 8 in 2024-04-14T15:23:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:52 smithi045 ceph-mon[37795]: pgmap v930: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:52.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:52 smithi045 ceph-mon[37795]: osdmap e605: 8 total, 8 up, 8 in 2024-04-14T15:23:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:52 smithi071 ceph-mon[33150]: pgmap v930: 204 pgs: 204 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:52.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:52 smithi071 ceph-mon[33150]: osdmap e605: 8 total, 8 up, 8 in 2024-04-14T15:23:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:54 smithi071 ceph-mon[33150]: pgmap v933: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:54 smithi071 ceph-mon[33150]: osdmap e606: 8 total, 8 up, 8 in 2024-04-14T15:23:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:54.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:54 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:54.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:23:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:23:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[29624]: pgmap v933: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[29624]: osdmap e606: 8 total, 8 up, 8 in 2024-04-14T15:23:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:54.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[37795]: pgmap v933: 172 pgs: 172 active+clean; 582 KiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:23:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[37795]: osdmap e606: 8 total, 8 up, 8 in 2024-04-14T15:23:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:54.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:54 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:23:55.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:55.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[29624]: osdmap e607: 8 total, 8 up, 8 in 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[37795]: osdmap e607: 8 total, 8 up, 8 in 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:55.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:23:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:23:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:55 smithi071 ceph-mon[33150]: osdmap e607: 8 total, 8 up, 8 in 2024-04-14T15:23:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:23:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[29624]: pgmap v936: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:56.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[29624]: osdmap e608: 8 total, 8 up, 8 in 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[37795]: pgmap v936: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[37795]: osdmap e608: 8 total, 8 up, 8 in 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:56 smithi071 ceph-mon[33150]: pgmap v936: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:23:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:56 smithi071 ceph-mon[33150]: osdmap e608: 8 total, 8 up, 8 in 2024-04-14T15:23:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:23:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:23:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:23:57.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: osdmap e609: 8 total, 8 up, 8 in 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]': finished 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: osdmap e609: 8 total, 8 up, 8 in 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:57.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]': finished 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: osdmap e609: 8 total, 8 up, 8 in 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]: dispatch 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:23:57.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-122", "mode": "writeback"}]': finished 2024-04-14T15:23:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[29624]: pgmap v939: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:58.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[29624]: osdmap e610: 8 total, 8 up, 8 in 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[37795]: pgmap v939: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[37795]: osdmap e610: 8 total, 8 up, 8 in 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:58.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:58 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:58 smithi071 ceph-mon[33150]: pgmap v939: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:23:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:58 smithi071 ceph-mon[33150]: osdmap e610: 8 total, 8 up, 8 in 2024-04-14T15:23:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:58.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:58 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: osdmap e611: 8 total, 8 up, 8 in 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:59.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: osdmap e611: 8 total, 8 up, 8 in 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:59.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:23:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: osdmap e611: 8 total, 8 up, 8 in 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]: dispatch 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:23:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:23:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:00.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[29624]: pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-14T15:24:00.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[29624]: osdmap e612: 8 total, 8 up, 8 in 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[37795]: pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:24:00.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:00 smithi045 ceph-mon[37795]: osdmap e612: 8 total, 8 up, 8 in 2024-04-14T15:24:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:00 smithi071 ceph-mon[33150]: pgmap v942: 204 pgs: 204 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-14T15:24:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:00 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:00 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-122"}]': finished 2024-04-14T15:24:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:00 smithi071 ceph-mon[33150]: osdmap e612: 8 total, 8 up, 8 in 2024-04-14T15:24:01.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:01 smithi045 ceph-mon[37795]: osdmap e613: 8 total, 8 up, 8 in 2024-04-14T15:24:01.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:01 smithi045 ceph-mon[29624]: osdmap e613: 8 total, 8 up, 8 in 2024-04-14T15:24:01.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:01 smithi071 ceph-mon[33150]: osdmap e613: 8 total, 8 up, 8 in 2024-04-14T15:24:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[29624]: pgmap v945: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:24:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[29624]: osdmap e614: 8 total, 8 up, 8 in 2024-04-14T15:24:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[37795]: pgmap v945: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:24:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[37795]: osdmap e614: 8 total, 8 up, 8 in 2024-04-14T15:24:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:02 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:02 smithi071 ceph-mon[33150]: pgmap v945: 172 pgs: 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-14T15:24:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:02 smithi071 ceph-mon[33150]: osdmap e614: 8 total, 8 up, 8 in 2024-04-14T15:24:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:02 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:03 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:03 smithi045 ceph-mon[37795]: osdmap e615: 8 total, 8 up, 8 in 2024-04-14T15:24:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:03 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:03 smithi045 ceph-mon[29624]: osdmap e615: 8 total, 8 up, 8 in 2024-04-14T15:24:03.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:03 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:03.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:03 smithi071 ceph-mon[33150]: osdmap e615: 8 total, 8 up, 8 in 2024-04-14T15:24:04.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:04 smithi071 ceph-mon[33150]: pgmap v948: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:04 smithi071 ceph-mon[33150]: osdmap e616: 8 total, 8 up, 8 in 2024-04-14T15:24:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:04.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[29624]: pgmap v948: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[29624]: osdmap e616: 8 total, 8 up, 8 in 2024-04-14T15:24:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:04.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[37795]: pgmap v948: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[37795]: osdmap e616: 8 total, 8 up, 8 in 2024-04-14T15:24:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[29624]: osdmap e617: 8 total, 8 up, 8 in 2024-04-14T15:24:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[37795]: osdmap e617: 8 total, 8 up, 8 in 2024-04-14T15:24:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:05.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:05 smithi071 ceph-mon[33150]: osdmap e617: 8 total, 8 up, 8 in 2024-04-14T15:24:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[29624]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[29624]: osdmap e618: 8 total, 8 up, 8 in 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[37795]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[37795]: osdmap e618: 8 total, 8 up, 8 in 2024-04-14T15:24:06.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:06 smithi071 ceph-mon[33150]: pgmap v951: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:06 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:06 smithi071 ceph-mon[33150]: osdmap e618: 8 total, 8 up, 8 in 2024-04-14T15:24:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:06 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]: dispatch 2024-04-14T15:24:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]': finished 2024-04-14T15:24:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[37795]: osdmap e619: 8 total, 8 up, 8 in 2024-04-14T15:24:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]': finished 2024-04-14T15:24:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:07 smithi045 ceph-mon[29624]: osdmap e619: 8 total, 8 up, 8 in 2024-04-14T15:24:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:07 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:07 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-124", "mode": "writeback"}]': finished 2024-04-14T15:24:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:07 smithi071 ceph-mon[33150]: osdmap e619: 8 total, 8 up, 8 in 2024-04-14T15:24:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[29624]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[29624]: osdmap e620: 8 total, 8 up, 8 in 2024-04-14T15:24:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[37795]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[37795]: osdmap e620: 8 total, 8 up, 8 in 2024-04-14T15:24:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:08.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:08 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:08.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:08 smithi071 ceph-mon[33150]: pgmap v954: 204 pgs: 204 active+clean; 583 KiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-14T15:24:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:08 smithi071 ceph-mon[33150]: osdmap e620: 8 total, 8 up, 8 in 2024-04-14T15:24:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:08 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:09.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:09.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[37795]: osdmap e621: 8 total, 8 up, 8 in 2024-04-14T15:24:09.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:09.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[29624]: osdmap e621: 8 total, 8 up, 8 in 2024-04-14T15:24:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:09.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:09 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:09.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:09 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:09 smithi071 ceph-mon[33150]: osdmap e621: 8 total, 8 up, 8 in 2024-04-14T15:24:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:09 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]: dispatch 2024-04-14T15:24:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[37795]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:24:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[37795]: osdmap e622: 8 total, 8 up, 8 in 2024-04-14T15:24:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[37795]: osdmap e623: 8 total, 8 up, 8 in 2024-04-14T15:24:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[29624]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:24:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[29624]: osdmap e622: 8 total, 8 up, 8 in 2024-04-14T15:24:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:10 smithi045 ceph-mon[29624]: osdmap e623: 8 total, 8 up, 8 in 2024-04-14T15:24:10.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:10 smithi071 ceph-mon[33150]: pgmap v957: 204 pgs: 204 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-14T15:24:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:10 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:10 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-124"}]': finished 2024-04-14T15:24:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:10 smithi071 ceph-mon[33150]: osdmap e622: 8 total, 8 up, 8 in 2024-04-14T15:24:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:10 smithi071 ceph-mon[33150]: osdmap e623: 8 total, 8 up, 8 in 2024-04-14T15:24:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[37795]: pgmap v960: 172 pgs: 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[37795]: osdmap e624: 8 total, 8 up, 8 in 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[29624]: pgmap v960: 172 pgs: 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[29624]: osdmap e624: 8 total, 8 up, 8 in 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:12 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:12.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:12 smithi071 ceph-mon[33150]: pgmap v960: 172 pgs: 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:12 smithi071 ceph-mon[33150]: osdmap e624: 8 total, 8 up, 8 in 2024-04-14T15:24:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:12 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[37795]: osdmap e625: 8 total, 8 up, 8 in 2024-04-14T15:24:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[29624]: osdmap e625: 8 total, 8 up, 8 in 2024-04-14T15:24:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:13 smithi071 ceph-mon[33150]: osdmap e625: 8 total, 8 up, 8 in 2024-04-14T15:24:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:14.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: pgmap v963: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: osdmap e626: 8 total, 8 up, 8 in 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:14.358 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: pgmap v963: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:14.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: osdmap e626: 8 total, 8 up, 8 in 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: pgmap v963: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: osdmap e626: 8 total, 8 up, 8 in 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[37795]: osdmap e627: 8 total, 8 up, 8 in 2024-04-14T15:24:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[29624]: osdmap e627: 8 total, 8 up, 8 in 2024-04-14T15:24:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:15.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:15.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:15 smithi071 ceph-mon[33150]: osdmap e627: 8 total, 8 up, 8 in 2024-04-14T15:24:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]: dispatch 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]': finished 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: osdmap e628: 8 total, 8 up, 8 in 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]': finished 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: osdmap e628: 8 total, 8 up, 8 in 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: pgmap v966: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-126", "mode": "writeback"}]': finished 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: osdmap e628: 8 total, 8 up, 8 in 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[37795]: osdmap e629: 8 total, 8 up, 8 in 2024-04-14T15:24:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[29624]: osdmap e629: 8 total, 8 up, 8 in 2024-04-14T15:24:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:17 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:17.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:17 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:17 smithi071 ceph-mon[33150]: osdmap e629: 8 total, 8 up, 8 in 2024-04-14T15:24:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:17 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]: dispatch 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9060 ms) 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6050 ms) 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5077 ms) 2024-04-14T15:24:18.186 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25984 ms) 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14038 ms) 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12094 ms) 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 00000010 69 73 |is| 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 00000012 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12042 ms) 2024-04-14T15:24:18.187 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: require_osd_release = squid 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6057 ms) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (555318 ms total) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1266 ms) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10795 ms) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18087 ms) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30151 ms total) 2024-04-14T15:24:18.188 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7051 ms) 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8044 ms) 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: waiting for scrub... 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: done waiting 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24540 ms) 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9981 ms) 2024-04-14T15:24:18.189 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7973 ms) 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8047 ms) 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (9946 ms) 2024-04-14T15:24:18.190 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-04-14T15:24:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[29624]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:18.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[29624]: osdmap e630: 8 total, 8 up, 8 in 2024-04-14T15:24:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[37795]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:18.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:18 smithi045 ceph-mon[37795]: osdmap e630: 8 total, 8 up, 8 in 2024-04-14T15:24:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:18 smithi071 ceph-mon[33150]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:18 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:18 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-126"}]': finished 2024-04-14T15:24:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:18 smithi071 ceph-mon[33150]: osdmap e630: 8 total, 8 up, 8 in 2024-04-14T15:24:19.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:19 smithi045 ceph-mon[29624]: osdmap e631: 8 total, 8 up, 8 in 2024-04-14T15:24:19.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:19 smithi045 ceph-mon[37795]: osdmap e631: 8 total, 8 up, 8 in 2024-04-14T15:24:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:19 smithi071 ceph-mon[33150]: osdmap e631: 8 total, 8 up, 8 in 2024-04-14T15:24:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[37795]: pgmap v972: 172 pgs: 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[37795]: osdmap e632: 8 total, 8 up, 8 in 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[29624]: pgmap v972: 172 pgs: 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[29624]: osdmap e632: 8 total, 8 up, 8 in 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:20 smithi071 ceph-mon[33150]: pgmap v972: 172 pgs: 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:20 smithi071 ceph-mon[33150]: osdmap e632: 8 total, 8 up, 8 in 2024-04-14T15:24:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:21.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: osdmap e633: 8 total, 8 up, 8 in 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: osdmap e634: 8 total, 8 up, 8 in 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: osdmap e633: 8 total, 8 up, 8 in 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: osdmap e634: 8 total, 8 up, 8 in 2024-04-14T15:24:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:21.498 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: osdmap e633: 8 total, 8 up, 8 in 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: osdmap e634: 8 total, 8 up, 8 in 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[29624]: pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[29624]: osdmap e635: 8 total, 8 up, 8 in 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[37795]: pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[37795]: osdmap e635: 8 total, 8 up, 8 in 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:22.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:22 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:22 smithi071 ceph-mon[33150]: pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:22 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:22 smithi071 ceph-mon[33150]: osdmap e635: 8 total, 8 up, 8 in 2024-04-14T15:24:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:22.508 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:22 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]: dispatch 2024-04-14T15:24:23.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:23.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]': finished 2024-04-14T15:24:23.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[29624]: osdmap e636: 8 total, 8 up, 8 in 2024-04-14T15:24:23.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]': finished 2024-04-14T15:24:23.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:23 smithi045 ceph-mon[37795]: osdmap e636: 8 total, 8 up, 8 in 2024-04-14T15:24:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:23 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:23 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-128", "mode": "writeback"}]': finished 2024-04-14T15:24:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:23 smithi071 ceph-mon[33150]: osdmap e636: 8 total, 8 up, 8 in 2024-04-14T15:24:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:24 smithi045 ceph-mon[29624]: pgmap v978: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:24.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:24 smithi045 ceph-mon[37795]: pgmap v978: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:24 smithi071 ceph-mon[33150]: pgmap v978: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:24.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:25.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:26 smithi045 ceph-mon[29624]: pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 463 B/s wr, 1 op/s 2024-04-14T15:24:26.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:26 smithi045 ceph-mon[37795]: pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 463 B/s wr, 1 op/s 2024-04-14T15:24:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:26 smithi071 ceph-mon[33150]: pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 463 B/s wr, 1 op/s 2024-04-14T15:24:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:24:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:24:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:24:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:27.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:24:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:24:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:24:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:27 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:27.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:27 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:24:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:27 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:24:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:27 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:24:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:27 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[37795]: pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-04-14T15:24:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[29624]: pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-04-14T15:24:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:28.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:28 smithi071 ceph-mon[33150]: pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-04-14T15:24:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:29 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:29 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:29.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:29 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:29 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:29 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:29 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[37795]: pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:30.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[37795]: osdmap e637: 8 total, 8 up, 8 in 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[29624]: pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[29624]: osdmap e637: 8 total, 8 up, 8 in 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:30.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:30 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:30 smithi071 ceph-mon[33150]: pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:30 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:30 smithi071 ceph-mon[33150]: osdmap e637: 8 total, 8 up, 8 in 2024-04-14T15:24:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:30 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]: dispatch 2024-04-14T15:24:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:31.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[37795]: osdmap e638: 8 total, 8 up, 8 in 2024-04-14T15:24:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:31.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:31 smithi045 ceph-mon[29624]: osdmap e638: 8 total, 8 up, 8 in 2024-04-14T15:24:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:31 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:31 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-128"}]': finished 2024-04-14T15:24:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:31 smithi071 ceph-mon[33150]: osdmap e638: 8 total, 8 up, 8 in 2024-04-14T15:24:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[29624]: pgmap v985: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[29624]: osdmap e639: 8 total, 8 up, 8 in 2024-04-14T15:24:32.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[37795]: pgmap v985: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:32 smithi045 ceph-mon[37795]: osdmap e639: 8 total, 8 up, 8 in 2024-04-14T15:24:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:32 smithi071 ceph-mon[33150]: pgmap v985: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 276 B/s wr, 2 op/s 2024-04-14T15:24:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:32 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:32 smithi071 ceph-mon[33150]: osdmap e639: 8 total, 8 up, 8 in 2024-04-14T15:24:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:33 smithi071 ceph-mon[33150]: osdmap e640: 8 total, 8 up, 8 in 2024-04-14T15:24:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:33 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:33 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[29624]: osdmap e640: 8 total, 8 up, 8 in 2024-04-14T15:24:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[37795]: osdmap e640: 8 total, 8 up, 8 in 2024-04-14T15:24:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:33 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-mon[33150]: pgmap v988: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-mon[33150]: osdmap e641: 8 total, 8 up, 8 in 2024-04-14T15:24:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:34.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:34.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[29624]: pgmap v988: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:34.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:34.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[29624]: osdmap e641: 8 total, 8 up, 8 in 2024-04-14T15:24:34.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:34.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:34.649 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[37795]: pgmap v988: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:34.649 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:34.649 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[37795]: osdmap e641: 8 total, 8 up, 8 in 2024-04-14T15:24:34.649 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:34.649 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:34 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: osdmap e642: 8 total, 8 up, 8 in 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:35 smithi071 ceph-mon[33150]: osdmap e643: 8 total, 8 up, 8 in 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: osdmap e642: 8 total, 8 up, 8 in 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[37795]: osdmap e643: 8 total, 8 up, 8 in 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: osdmap e642: 8 total, 8 up, 8 in 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:35.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:35 smithi045 ceph-mon[29624]: osdmap e643: 8 total, 8 up, 8 in 2024-04-14T15:24:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:36 smithi071 ceph-mon[33150]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:24:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:36 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:36 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:36 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]': finished 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[37795]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]': finished 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[29624]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]: dispatch 2024-04-14T15:24:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:36.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-130", "mode": "writeback"}]': finished 2024-04-14T15:24:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:37 smithi071 ceph-mon[33150]: osdmap e644: 8 total, 8 up, 8 in 2024-04-14T15:24:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:37.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[37795]: osdmap e644: 8 total, 8 up, 8 in 2024-04-14T15:24:37.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:37.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:37.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[29624]: osdmap e644: 8 total, 8 up, 8 in 2024-04-14T15:24:37.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:37.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:38 smithi071 ceph-mon[33150]: pgmap v994: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:38 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:38 smithi071 ceph-mon[33150]: osdmap e645: 8 total, 8 up, 8 in 2024-04-14T15:24:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:38 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[37795]: pgmap v994: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[37795]: osdmap e645: 8 total, 8 up, 8 in 2024-04-14T15:24:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[29624]: pgmap v994: 204 pgs: 204 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[29624]: osdmap e645: 8 total, 8 up, 8 in 2024-04-14T15:24:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:38 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]: dispatch 2024-04-14T15:24:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:39 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:39 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:39 smithi071 ceph-mon[33150]: osdmap e646: 8 total, 8 up, 8 in 2024-04-14T15:24:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[37795]: osdmap e646: 8 total, 8 up, 8 in 2024-04-14T15:24:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-130"}]': finished 2024-04-14T15:24:39.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:39 smithi045 ceph-mon[29624]: osdmap e646: 8 total, 8 up, 8 in 2024-04-14T15:24:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:40 smithi071 ceph-mon[33150]: pgmap v997: 204 pgs: 204 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:24:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:40 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:40 smithi071 ceph-mon[33150]: osdmap e647: 8 total, 8 up, 8 in 2024-04-14T15:24:40.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[29624]: pgmap v997: 204 pgs: 204 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:24:40.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:40.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[29624]: osdmap e647: 8 total, 8 up, 8 in 2024-04-14T15:24:40.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[37795]: pgmap v997: 204 pgs: 204 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-14T15:24:40.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:40.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:40 smithi045 ceph-mon[37795]: osdmap e647: 8 total, 8 up, 8 in 2024-04-14T15:24:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:41 smithi071 ceph-mon[33150]: osdmap e648: 8 total, 8 up, 8 in 2024-04-14T15:24:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:41.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:41 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[37795]: osdmap e648: 8 total, 8 up, 8 in 2024-04-14T15:24:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:41.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[29624]: osdmap e648: 8 total, 8 up, 8 in 2024-04-14T15:24:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:41 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:42 smithi071 ceph-mon[33150]: pgmap v1000: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:42 smithi071 ceph-mon[33150]: osdmap e649: 8 total, 8 up, 8 in 2024-04-14T15:24:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:42.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[29624]: pgmap v1000: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:42.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:42.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[29624]: osdmap e649: 8 total, 8 up, 8 in 2024-04-14T15:24:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[37795]: pgmap v1000: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-04-14T15:24:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[37795]: osdmap e649: 8 total, 8 up, 8 in 2024-04-14T15:24:42.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:42.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: osdmap e650: 8 total, 8 up, 8 in 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:43.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: osdmap e650: 8 total, 8 up, 8 in 2024-04-14T15:24:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: osdmap e650: 8 total, 8 up, 8 in 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:44.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: osdmap e651: 8 total, 8 up, 8 in 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]': finished 2024-04-14T15:24:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:44 smithi071 ceph-mon[33150]: osdmap e652: 8 total, 8 up, 8 in 2024-04-14T15:24:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: osdmap e651: 8 total, 8 up, 8 in 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]': finished 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[29624]: osdmap e652: 8 total, 8 up, 8 in 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: pgmap v1003: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: osdmap e651: 8 total, 8 up, 8 in 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]: dispatch 2024-04-14T15:24:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:44.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-132", "mode": "writeback"}]': finished 2024-04-14T15:24:44.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:44 smithi045 ceph-mon[37795]: osdmap e652: 8 total, 8 up, 8 in 2024-04-14T15:24:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:45 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:45.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:45 smithi071 ceph-mon[33150]: osdmap e653: 8 total, 8 up, 8 in 2024-04-14T15:24:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[29624]: osdmap e653: 8 total, 8 up, 8 in 2024-04-14T15:24:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:24:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:45 smithi045 ceph-mon[37795]: osdmap e653: 8 total, 8 up, 8 in 2024-04-14T15:24:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:46 smithi071 ceph-mon[33150]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:46 smithi071 ceph-mon[33150]: osdmap e654: 8 total, 8 up, 8 in 2024-04-14T15:24:46.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:46 smithi045 ceph-mon[37795]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:46.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:46 smithi045 ceph-mon[37795]: osdmap e654: 8 total, 8 up, 8 in 2024-04-14T15:24:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:46 smithi045 ceph-mon[29624]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:46 smithi045 ceph-mon[29624]: osdmap e654: 8 total, 8 up, 8 in 2024-04-14T15:24:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:47.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:47 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:47.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:47 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:47 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:48 smithi071 ceph-mon[33150]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:48 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:48 smithi071 ceph-mon[33150]: osdmap e655: 8 total, 8 up, 8 in 2024-04-14T15:24:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:48 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[29624]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[29624]: osdmap e655: 8 total, 8 up, 8 in 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:48.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[37795]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[37795]: osdmap e655: 8 total, 8 up, 8 in 2024-04-14T15:24:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:48.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:48 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:49.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:49.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:49 smithi071 ceph-mon[33150]: osdmap e656: 8 total, 8 up, 8 in 2024-04-14T15:24:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:49 smithi045 ceph-mon[29624]: osdmap e656: 8 total, 8 up, 8 in 2024-04-14T15:24:49.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:49.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:49 smithi045 ceph-mon[37795]: osdmap e656: 8 total, 8 up, 8 in 2024-04-14T15:24:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:50 smithi071 ceph-mon[33150]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:50 smithi071 ceph-mon[33150]: osdmap e657: 8 total, 8 up, 8 in 2024-04-14T15:24:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[29624]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[29624]: osdmap e657: 8 total, 8 up, 8 in 2024-04-14T15:24:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:50.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[37795]: pgmap v1012: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[37795]: osdmap e657: 8 total, 8 up, 8 in 2024-04-14T15:24:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:51.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:51.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[29624]: osdmap e658: 8 total, 8 up, 8 in 2024-04-14T15:24:51.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:51.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[37795]: osdmap e658: 8 total, 8 up, 8 in 2024-04-14T15:24:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:51 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:51 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:51 smithi071 ceph-mon[33150]: osdmap e658: 8 total, 8 up, 8 in 2024-04-14T15:24:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:51 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]: dispatch 2024-04-14T15:24:52.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[29624]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:52.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:52.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[29624]: osdmap e659: 8 total, 8 up, 8 in 2024-04-14T15:24:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[37795]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:52 smithi045 ceph-mon[37795]: osdmap e659: 8 total, 8 up, 8 in 2024-04-14T15:24:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:52 smithi071 ceph-mon[33150]: pgmap v1015: 204 pgs: 204 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-14T15:24:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:52 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:24:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-132"}]': finished 2024-04-14T15:24:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:52 smithi071 ceph-mon[33150]: osdmap e659: 8 total, 8 up, 8 in 2024-04-14T15:24:53.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[29624]: osdmap e660: 8 total, 8 up, 8 in 2024-04-14T15:24:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[29624]: pgmap v1018: 172 pgs: 172 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[29624]: osdmap e661: 8 total, 8 up, 8 in 2024-04-14T15:24:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:53.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[37795]: osdmap e660: 8 total, 8 up, 8 in 2024-04-14T15:24:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[37795]: pgmap v1018: 172 pgs: 172 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[37795]: osdmap e661: 8 total, 8 up, 8 in 2024-04-14T15:24:53.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:53.749 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:53 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:53 smithi071 ceph-mon[33150]: osdmap e660: 8 total, 8 up, 8 in 2024-04-14T15:24:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:53 smithi071 ceph-mon[33150]: pgmap v1018: 172 pgs: 172 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:24:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:53 smithi071 ceph-mon[33150]: osdmap e661: 8 total, 8 up, 8 in 2024-04-14T15:24:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:53 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:24:54.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:24:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:24:55.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:55.753 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[37795]: osdmap e662: 8 total, 8 up, 8 in 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[29624]: osdmap e662: 8 total, 8 up, 8 in 2024-04-14T15:24:55.754 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:55.755 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:24:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:24:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:55 smithi071 ceph-mon[33150]: osdmap e662: 8 total, 8 up, 8 in 2024-04-14T15:24:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:24:56.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[37795]: pgmap v1021: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[37795]: osdmap e663: 8 total, 8 up, 8 in 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[29624]: pgmap v1021: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[29624]: osdmap e663: 8 total, 8 up, 8 in 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:56 smithi071 ceph-mon[33150]: pgmap v1021: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-04-14T15:24:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:24:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:56 smithi071 ceph-mon[33150]: osdmap e663: 8 total, 8 up, 8 in 2024-04-14T15:24:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:24:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:24:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: osdmap e664: 8 total, 8 up, 8 in 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: pgmap v1024: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]': finished 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[37795]: osdmap e665: 8 total, 8 up, 8 in 2024-04-14T15:24:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: osdmap e664: 8 total, 8 up, 8 in 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: pgmap v1024: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]': finished 2024-04-14T15:24:57.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:57 smithi045 ceph-mon[29624]: osdmap e665: 8 total, 8 up, 8 in 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: osdmap e664: 8 total, 8 up, 8 in 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]: dispatch 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: pgmap v1024: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-134", "mode": "writeback"}]': finished 2024-04-14T15:24:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:57 smithi071 ceph-mon[33150]: osdmap e665: 8 total, 8 up, 8 in 2024-04-14T15:24:58.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:58.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:58.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:58 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:58 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:24:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:24:58.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:24:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[37795]: osdmap e666: 8 total, 8 up, 8 in 2024-04-14T15:24:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[37795]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[29624]: osdmap e666: 8 total, 8 up, 8 in 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:24:59 smithi045 ceph-mon[29624]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-14T15:24:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:24:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:59 smithi071 ceph-mon[33150]: osdmap e666: 8 total, 8 up, 8 in 2024-04-14T15:24:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:59 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:59 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]: dispatch 2024-04-14T15:24:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:24:59 smithi071 ceph-mon[33150]: pgmap v1027: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-04-14T15:25:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:25:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[29624]: osdmap e667: 8 total, 8 up, 8 in 2024-04-14T15:25:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[29624]: osdmap e668: 8 total, 8 up, 8 in 2024-04-14T15:25:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:25:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[37795]: osdmap e667: 8 total, 8 up, 8 in 2024-04-14T15:25:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:00 smithi045 ceph-mon[37795]: osdmap e668: 8 total, 8 up, 8 in 2024-04-14T15:25:00.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:00 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:00 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-134"}]': finished 2024-04-14T15:25:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:00 smithi071 ceph-mon[33150]: osdmap e667: 8 total, 8 up, 8 in 2024-04-14T15:25:00.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:00 smithi071 ceph-mon[33150]: osdmap e668: 8 total, 8 up, 8 in 2024-04-14T15:25:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[37795]: pgmap v1030: 172 pgs: 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[37795]: osdmap e669: 8 total, 8 up, 8 in 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[29624]: pgmap v1030: 172 pgs: 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[29624]: osdmap e669: 8 total, 8 up, 8 in 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:02.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:02 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:02 smithi071 ceph-mon[33150]: pgmap v1030: 172 pgs: 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-14T15:25:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:02 smithi071 ceph-mon[33150]: osdmap e669: 8 total, 8 up, 8 in 2024-04-14T15:25:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:02 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:02 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[37795]: osdmap e670: 8 total, 8 up, 8 in 2024-04-14T15:25:03.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:03.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[29624]: osdmap e670: 8 total, 8 up, 8 in 2024-04-14T15:25:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:03.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:03 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:03 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:03 smithi071 ceph-mon[33150]: osdmap e670: 8 total, 8 up, 8 in 2024-04-14T15:25:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:03 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[37795]: pgmap v1033: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:25:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[37795]: osdmap e671: 8 total, 8 up, 8 in 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[29624]: pgmap v1033: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[29624]: osdmap e671: 8 total, 8 up, 8 in 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:04 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-mon[33150]: pgmap v1033: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-04-14T15:25:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-mon[33150]: osdmap e671: 8 total, 8 up, 8 in 2024-04-14T15:25:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:04.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[37795]: osdmap e672: 8 total, 8 up, 8 in 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[29624]: osdmap e672: 8 total, 8 up, 8 in 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:05.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:05 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:05 smithi071 ceph-mon[33150]: osdmap e672: 8 total, 8 up, 8 in 2024-04-14T15:25:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:05 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]: dispatch 2024-04-14T15:25:06.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: pgmap v1036: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]': finished 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: osdmap e673: 8 total, 8 up, 8 in 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: pgmap v1036: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]': finished 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: osdmap e673: 8 total, 8 up, 8 in 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: pgmap v1036: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-136", "mode": "writeback"}]': finished 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: osdmap e673: 8 total, 8 up, 8 in 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:06 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:07.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[37795]: osdmap e674: 8 total, 8 up, 8 in 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[29624]: osdmap e674: 8 total, 8 up, 8 in 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:07.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:07 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:07 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:07 smithi071 ceph-mon[33150]: osdmap e674: 8 total, 8 up, 8 in 2024-04-14T15:25:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:07 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]: dispatch 2024-04-14T15:25:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[37795]: pgmap v1039: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[37795]: osdmap e675: 8 total, 8 up, 8 in 2024-04-14T15:25:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[29624]: pgmap v1039: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:08.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:08 smithi045 ceph-mon[29624]: osdmap e675: 8 total, 8 up, 8 in 2024-04-14T15:25:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:08 smithi071 ceph-mon[33150]: pgmap v1039: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:08 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:08 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-136"}]': finished 2024-04-14T15:25:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:08 smithi071 ceph-mon[33150]: osdmap e675: 8 total, 8 up, 8 in 2024-04-14T15:25:09.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:09 smithi045 ceph-mon[37795]: osdmap e676: 8 total, 8 up, 8 in 2024-04-14T15:25:09.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:09 smithi045 ceph-mon[29624]: osdmap e676: 8 total, 8 up, 8 in 2024-04-14T15:25:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:09 smithi071 ceph-mon[33150]: osdmap e676: 8 total, 8 up, 8 in 2024-04-14T15:25:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[29624]: pgmap v1042: 172 pgs: 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:10.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[29624]: osdmap e677: 8 total, 8 up, 8 in 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[37795]: pgmap v1042: 172 pgs: 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[37795]: osdmap e677: 8 total, 8 up, 8 in 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:10.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:10 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:10 smithi071 ceph-mon[33150]: pgmap v1042: 172 pgs: 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:10 smithi071 ceph-mon[33150]: osdmap e677: 8 total, 8 up, 8 in 2024-04-14T15:25:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:10 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:11.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:11.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[37795]: osdmap e678: 8 total, 8 up, 8 in 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[29624]: osdmap e678: 8 total, 8 up, 8 in 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:11.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:11 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:11 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:11 smithi071 ceph-mon[33150]: osdmap e678: 8 total, 8 up, 8 in 2024-04-14T15:25:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:11 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[37795]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[37795]: osdmap e679: 8 total, 8 up, 8 in 2024-04-14T15:25:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[29624]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[29624]: osdmap e679: 8 total, 8 up, 8 in 2024-04-14T15:25:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:12.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:12 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:12 smithi071 ceph-mon[33150]: pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-04-14T15:25:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:12 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:12 smithi071 ceph-mon[33150]: osdmap e679: 8 total, 8 up, 8 in 2024-04-14T15:25:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:12 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: osdmap e680: 8 total, 8 up, 8 in 2024-04-14T15:25:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[37795]: osdmap e681: 8 total, 8 up, 8 in 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: osdmap e680: 8 total, 8 up, 8 in 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:13.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:13 smithi045 ceph-mon[29624]: osdmap e681: 8 total, 8 up, 8 in 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: osdmap e680: 8 total, 8 up, 8 in 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:13.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:13 smithi071 ceph-mon[33150]: osdmap e681: 8 total, 8 up, 8 in 2024-04-14T15:25:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[29624]: osdmap e682: 8 total, 8 up, 8 in 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:25:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:14 smithi045 ceph-mon[37795]: osdmap e682: 8 total, 8 up, 8 in 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-04-14T15:25:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-mon[33150]: osdmap e682: 8 total, 8 up, 8 in 2024-04-14T15:25:14.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:15.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:15.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[37795]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[29624]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:16 smithi071 ceph-mon[33150]: pgmap v1051: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]: dispatch 2024-04-14T15:25:16.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:17 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]': finished 2024-04-14T15:25:17.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:17 smithi045 ceph-mon[37795]: osdmap e683: 8 total, 8 up, 8 in 2024-04-14T15:25:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:17 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]': finished 2024-04-14T15:25:17.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:17 smithi045 ceph-mon[29624]: osdmap e683: 8 total, 8 up, 8 in 2024-04-14T15:25:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:17 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-138"}]': finished 2024-04-14T15:25:17.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:17 smithi071 ceph-mon[33150]: osdmap e683: 8 total, 8 up, 8 in 2024-04-14T15:25:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:18 smithi071 ceph-mon[33150]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:18 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:18.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:18 smithi071 ceph-mon[33150]: osdmap e684: 8 total, 8 up, 8 in 2024-04-14T15:25:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[37795]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:18.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[37795]: osdmap e684: 8 total, 8 up, 8 in 2024-04-14T15:25:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[29624]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:18 smithi045 ceph-mon[29624]: osdmap e684: 8 total, 8 up, 8 in 2024-04-14T15:25:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:19 smithi071 ceph-mon[33150]: osdmap e685: 8 total, 8 up, 8 in 2024-04-14T15:25:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:19.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:19 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[37795]: osdmap e685: 8 total, 8 up, 8 in 2024-04-14T15:25:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[29624]: osdmap e685: 8 total, 8 up, 8 in 2024-04-14T15:25:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:19 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:20 smithi071 ceph-mon[33150]: pgmap v1056: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:20 smithi071 ceph-mon[33150]: osdmap e686: 8 total, 8 up, 8 in 2024-04-14T15:25:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:20.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[37795]: pgmap v1056: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[37795]: osdmap e686: 8 total, 8 up, 8 in 2024-04-14T15:25:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[29624]: pgmap v1056: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-14T15:25:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[29624]: osdmap e686: 8 total, 8 up, 8 in 2024-04-14T15:25:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:21 smithi071 ceph-mon[33150]: osdmap e687: 8 total, 8 up, 8 in 2024-04-14T15:25:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:21 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:21.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:21 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[29624]: osdmap e687: 8 total, 8 up, 8 in 2024-04-14T15:25:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:21.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[37795]: osdmap e687: 8 total, 8 up, 8 in 2024-04-14T15:25:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:21.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:21 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]: dispatch 2024-04-14T15:25:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:22 smithi071 ceph-mon[33150]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:22 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:25:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:22 smithi071 ceph-mon[33150]: osdmap e688: 8 total, 8 up, 8 in 2024-04-14T15:25:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:22 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:22.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:22 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[29624]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:25:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[29624]: osdmap e688: 8 total, 8 up, 8 in 2024-04-14T15:25:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[37795]: pgmap v1059: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_count","val": "3"}]': finished 2024-04-14T15:25:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[37795]: osdmap e688: 8 total, 8 up, 8 in 2024-04-14T15:25:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:22 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]: dispatch 2024-04-14T15:25:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:23 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:25:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:23 smithi071 ceph-mon[33150]: osdmap e689: 8 total, 8 up, 8 in 2024-04-14T15:25:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:23 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:23.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:23 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:23.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:25:23.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[37795]: osdmap e689: 8 total, 8 up, 8 in 2024-04-14T15:25:23.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:23.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:23.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_period","val": "3"}]': finished 2024-04-14T15:25:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[29624]: osdmap e689: 8 total, 8 up, 8 in 2024-04-14T15:25:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:23.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:23 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:24.609 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:24.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-mon[33150]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:24.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:24.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-mon[33150]: osdmap e690: 8 total, 8 up, 8 in 2024-04-14T15:25:24.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:24.610 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:24 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[37795]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[37795]: osdmap e690: 8 total, 8 up, 8 in 2024-04-14T15:25:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[29624]: pgmap v1062: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[29624]: osdmap e690: 8 total, 8 up, 8 in 2024-04-14T15:25:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:24.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:24 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-04-14T15:25:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:25 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:25:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:25 smithi071 ceph-mon[33150]: osdmap e691: 8 total, 8 up, 8 in 2024-04-14T15:25:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:25 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:25:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[37795]: osdmap e691: 8 total, 8 up, 8 in 2024-04-14T15:25:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-04-14T15:25:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[29624]: osdmap e691: 8 total, 8 up, 8 in 2024-04-14T15:25:25.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:25 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:25:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:26 smithi071 ceph-mon[33150]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:26.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:26 smithi045 ceph-mon[37795]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:26.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:26 smithi045 ceph-mon[29624]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:27.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:27 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:25:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:27 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:25:27.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:27 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:25:28.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:28 smithi071 ceph-mon[33150]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-04-14T15:25:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:25:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:25:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[29624]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-04-14T15:25:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:25:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:25:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:28.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[37795]: pgmap v1066: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-04-14T15:25:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:25:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:25:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:28.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:30 smithi071 ceph-mon[33150]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-04-14T15:25:30.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:30 smithi045 ceph-mon[29624]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-04-14T15:25:30.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:30 smithi045 ceph-mon[37795]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 6.5 KiB/s wr, 2 op/s 2024-04-14T15:25:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:32 smithi071 ceph-mon[33150]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-04-14T15:25:32.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:32 smithi045 ceph-mon[37795]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-04-14T15:25:32.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:32 smithi045 ceph-mon[29624]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.1 KiB/s wr, 1 op/s 2024-04-14T15:25:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:34 smithi071 ceph-mon[33150]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.4 KiB/s wr, 1 op/s 2024-04-14T15:25:34.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:34.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:34 smithi045 ceph-mon[29624]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.4 KiB/s wr, 1 op/s 2024-04-14T15:25:34.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:34 smithi045 ceph-mon[37795]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.4 KiB/s wr, 1 op/s 2024-04-14T15:25:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:35.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:36 smithi071 ceph-mon[33150]: pgmap v1070: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 990 B/s rd, 8.0 KiB/s wr, 2 op/s 2024-04-14T15:25:36.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:36 smithi045 ceph-mon[37795]: pgmap v1070: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 990 B/s rd, 8.0 KiB/s wr, 2 op/s 2024-04-14T15:25:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:36 smithi045 ceph-mon[29624]: pgmap v1070: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 990 B/s rd, 8.0 KiB/s wr, 2 op/s 2024-04-14T15:25:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:37.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:37.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:37.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]: dispatch 2024-04-14T15:25:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:38 smithi071 ceph-mon[33150]: pgmap v1071: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-04-14T15:25:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:38 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]': finished 2024-04-14T15:25:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:38 smithi071 ceph-mon[33150]: osdmap e692: 8 total, 8 up, 8 in 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[29624]: pgmap v1071: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]': finished 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[29624]: osdmap e692: 8 total, 8 up, 8 in 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[37795]: pgmap v1071: 204 pgs: 204 active+clean; 4.5 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-140"}]': finished 2024-04-14T15:25:38.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:38 smithi045 ceph-mon[37795]: osdmap e692: 8 total, 8 up, 8 in 2024-04-14T15:25:39.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:39 smithi071 ceph-mon[33150]: osdmap e693: 8 total, 8 up, 8 in 2024-04-14T15:25:39.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:39 smithi045 ceph-mon[37795]: osdmap e693: 8 total, 8 up, 8 in 2024-04-14T15:25:39.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:39 smithi045 ceph-mon[29624]: osdmap e693: 8 total, 8 up, 8 in 2024-04-14T15:25:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:40 smithi071 ceph-mon[33150]: pgmap v1074: 172 pgs: 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:40 smithi071 ceph-mon[33150]: osdmap e694: 8 total, 8 up, 8 in 2024-04-14T15:25:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:40.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[37795]: pgmap v1074: 172 pgs: 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:40.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[37795]: osdmap e694: 8 total, 8 up, 8 in 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[29624]: pgmap v1074: 172 pgs: 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[29624]: osdmap e694: 8 total, 8 up, 8 in 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:40.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[37795]: osdmap e695: 8 total, 8 up, 8 in 2024-04-14T15:25:41.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:41.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[29624]: osdmap e695: 8 total, 8 up, 8 in 2024-04-14T15:25:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:41.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:41 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:41 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:41 smithi071 ceph-mon[33150]: osdmap e695: 8 total, 8 up, 8 in 2024-04-14T15:25:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:41 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:41.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:41 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[37795]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[37795]: osdmap e696: 8 total, 8 up, 8 in 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[29624]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[29624]: osdmap e696: 8 total, 8 up, 8 in 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:42.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:42 smithi071 ceph-mon[33150]: pgmap v1077: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:42 smithi071 ceph-mon[33150]: osdmap e696: 8 total, 8 up, 8 in 2024-04-14T15:25:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:42.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:43.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[37795]: osdmap e697: 8 total, 8 up, 8 in 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[29624]: osdmap e697: 8 total, 8 up, 8 in 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:43.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:43 smithi071 ceph-mon[33150]: osdmap e697: 8 total, 8 up, 8 in 2024-04-14T15:25:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]: dispatch 2024-04-14T15:25:43.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:44.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]': finished 2024-04-14T15:25:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: osdmap e698: 8 total, 8 up, 8 in 2024-04-14T15:25:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:44.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:44.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]': finished 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: osdmap e698: 8 total, 8 up, 8 in 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: pgmap v1080: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 745 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-142", "mode": "writeback"}]': finished 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: osdmap e698: 8 total, 8 up, 8 in 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:25:44.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:45.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: osdmap e699: 8 total, 8 up, 8 in 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: osdmap e699: 8 total, 8 up, 8 in 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:45.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:45 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: osdmap e699: 8 total, 8 up, 8 in 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: pgmap v1083: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:45.858 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:45 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: osdmap e700: 8 total, 8 up, 8 in 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: osdmap e701: 8 total, 8 up, 8 in 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: osdmap e700: 8 total, 8 up, 8 in 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.729 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:46.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:46.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: osdmap e701: 8 total, 8 up, 8 in 2024-04-14T15:25:46.730 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: osdmap e700: 8 total, 8 up, 8 in 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: osdmap e701: 8 total, 8 up, 8 in 2024-04-14T15:25:46.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:25:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[29624]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:25:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[29624]: osdmap e702: 8 total, 8 up, 8 in 2024-04-14T15:25:47.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[37795]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[37795]: osdmap e702: 8 total, 8 up, 8 in 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:47.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:47 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:47 smithi071 ceph-mon[33150]: pgmap v1086: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-04-14T15:25:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:47 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:25:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:47 smithi071 ceph-mon[33150]: osdmap e702: 8 total, 8 up, 8 in 2024-04-14T15:25:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:47.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:47 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-04-14T15:25:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:25:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[29624]: osdmap e703: 8 total, 8 up, 8 in 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[29624]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[37795]: osdmap e703: 8 total, 8 up, 8 in 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:49 smithi045 ceph-mon[37795]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-04-14T15:25:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:49 smithi071 ceph-mon[33150]: osdmap e703: 8 total, 8 up, 8 in 2024-04-14T15:25:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-04-14T15:25:49.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:49 smithi071 ceph-mon[33150]: pgmap v1089: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:25:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[29624]: osdmap e704: 8 total, 8 up, 8 in 2024-04-14T15:25:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:25:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[37795]: osdmap e704: 8 total, 8 up, 8 in 2024-04-14T15:25:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:50.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-04-14T15:25:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:50 smithi071 ceph-mon[33150]: osdmap e704: 8 total, 8 up, 8 in 2024-04-14T15:25:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:50.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:51.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:51.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[37795]: osdmap e705: 8 total, 8 up, 8 in 2024-04-14T15:25:51.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[37795]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[29624]: osdmap e705: 8 total, 8 up, 8 in 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:51 smithi045 ceph-mon[29624]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:51 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:25:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:51 smithi071 ceph-mon[33150]: osdmap e705: 8 total, 8 up, 8 in 2024-04-14T15:25:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:51 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:51.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:51 smithi071 ceph-mon[33150]: pgmap v1092: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:25:52.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:52.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: osdmap e706: 8 total, 8 up, 8 in 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: osdmap e706: 8 total, 8 up, 8 in 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:52.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:52 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]': finished 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: osdmap e706: 8 total, 8 up, 8 in 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:52.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:52 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-142"}]: dispatch 2024-04-14T15:25:53.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:53 smithi045 ceph-mon[29624]: osdmap e707: 8 total, 8 up, 8 in 2024-04-14T15:25:53.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:53 smithi045 ceph-mon[29624]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:53.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:53 smithi045 ceph-mon[37795]: osdmap e707: 8 total, 8 up, 8 in 2024-04-14T15:25:53.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:53 smithi045 ceph-mon[37795]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:53.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:53 smithi071 ceph-mon[33150]: osdmap e707: 8 total, 8 up, 8 in 2024-04-14T15:25:53.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:53 smithi071 ceph-mon[33150]: pgmap v1095: 172 pgs: 172 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:25:54.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:25:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:25:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:54 smithi071 ceph-mon[33150]: osdmap e708: 8 total, 8 up, 8 in 2024-04-14T15:25:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:54 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:54.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[29624]: osdmap e708: 8 total, 8 up, 8 in 2024-04-14T15:25:54.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[37795]: osdmap e708: 8 total, 8 up, 8 in 2024-04-14T15:25:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:25:54.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:25:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[29624]: osdmap e709: 8 total, 8 up, 8 in 2024-04-14T15:25:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[29624]: pgmap v1098: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[37795]: osdmap e709: 8 total, 8 up, 8 in 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:55 smithi045 ceph-mon[37795]: pgmap v1098: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:25:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:55 smithi071 ceph-mon[33150]: osdmap e709: 8 total, 8 up, 8 in 2024-04-14T15:25:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:55 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:25:55.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:55 smithi071 ceph-mon[33150]: pgmap v1098: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:56.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[37795]: osdmap e710: 8 total, 8 up, 8 in 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[29624]: osdmap e710: 8 total, 8 up, 8 in 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:25:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:56 smithi071 ceph-mon[33150]: osdmap e710: 8 total, 8 up, 8 in 2024-04-14T15:25:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:56 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:56 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:25:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:25:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:25:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:25:57.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: osdmap e711: 8 total, 8 up, 8 in 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: pgmap v1101: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]': finished 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[29624]: osdmap e712: 8 total, 8 up, 8 in 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: osdmap e711: 8 total, 8 up, 8 in 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: pgmap v1101: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:57.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:57.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]': finished 2024-04-14T15:25:57.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:57 smithi045 ceph-mon[37795]: osdmap e712: 8 total, 8 up, 8 in 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: osdmap e711: 8 total, 8 up, 8 in 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]: dispatch 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: pgmap v1101: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-144", "mode": "readproxy"}]': finished 2024-04-14T15:25:57.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:57 smithi071 ceph-mon[33150]: osdmap e712: 8 total, 8 up, 8 in 2024-04-14T15:25:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:25:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:25:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:25:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:25:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:25:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:00 smithi071 ceph-mon[33150]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 243 B/s wr, 1 op/s 2024-04-14T15:26:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:00 smithi045 ceph-mon[29624]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 243 B/s wr, 1 op/s 2024-04-14T15:26:00.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:00 smithi045 ceph-mon[37795]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 243 B/s wr, 1 op/s 2024-04-14T15:26:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:02 smithi071 ceph-mon[33150]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-14T15:26:02.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:02 smithi045 ceph-mon[29624]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-14T15:26:02.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:02 smithi045 ceph-mon[37795]: pgmap v1104: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-04-14T15:26:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:04 smithi071 ceph-mon[33150]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 142 B/s wr, 0 op/s 2024-04-14T15:26:04.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:04.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:04 smithi045 ceph-mon[29624]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 142 B/s wr, 0 op/s 2024-04-14T15:26:04.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:04 smithi045 ceph-mon[37795]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 142 B/s wr, 0 op/s 2024-04-14T15:26:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:05.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:06 smithi071 ceph-mon[33150]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-04-14T15:26:06.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:06 smithi045 ceph-mon[37795]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-04-14T15:26:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:06 smithi045 ceph-mon[29624]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-04-14T15:26:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:08 smithi071 ceph-mon[33150]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:26:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:08 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[29624]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:26:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:08.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[37795]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-04-14T15:26:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:08 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:09 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:09 smithi071 ceph-mon[33150]: osdmap e713: 8 total, 8 up, 8 in 2024-04-14T15:26:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:09 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[37795]: osdmap e713: 8 total, 8 up, 8 in 2024-04-14T15:26:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[29624]: osdmap e713: 8 total, 8 up, 8 in 2024-04-14T15:26:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:09.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:09 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: osdmap e714: 8 total, 8 up, 8 in 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:10 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:26:10.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: osdmap e714: 8 total, 8 up, 8 in 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]': finished 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: osdmap e714: 8 total, 8 up, 8 in 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:10.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:10 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-144"}]: dispatch 2024-04-14T15:26:11.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:11 smithi071 ceph-mon[33150]: osdmap e715: 8 total, 8 up, 8 in 2024-04-14T15:26:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:11 smithi045 ceph-mon[37795]: osdmap e715: 8 total, 8 up, 8 in 2024-04-14T15:26:11.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:11 smithi045 ceph-mon[29624]: osdmap e715: 8 total, 8 up, 8 in 2024-04-14T15:26:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:12 smithi071 ceph-mon[33150]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:26:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:12 smithi071 ceph-mon[33150]: osdmap e716: 8 total, 8 up, 8 in 2024-04-14T15:26:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:12 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[37795]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:26:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[37795]: osdmap e716: 8 total, 8 up, 8 in 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[29624]: pgmap v1112: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[29624]: osdmap e716: 8 total, 8 up, 8 in 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:12 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:13.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:13.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: osdmap e717: 8 total, 8 up, 8 in 2024-04-14T15:26:13.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: osdmap e717: 8 total, 8 up, 8 in 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:13.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: osdmap e717: 8 total, 8 up, 8 in 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:13.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-mon[33150]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-mon[33150]: osdmap e718: 8 total, 8 up, 8 in 2024-04-14T15:26:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:14.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[29624]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[29624]: osdmap e718: 8 total, 8 up, 8 in 2024-04-14T15:26:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:14.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[37795]: pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[37795]: osdmap e718: 8 total, 8 up, 8 in 2024-04-14T15:26:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:14.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:14 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[37795]: osdmap e719: 8 total, 8 up, 8 in 2024-04-14T15:26:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[29624]: osdmap e719: 8 total, 8 up, 8 in 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:15.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:15 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi045-74757-111", "overlaypool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:15 smithi071 ceph-mon[33150]: osdmap e719: 8 total, 8 up, 8 in 2024-04-14T15:26:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:15 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]: dispatch 2024-04-14T15:26:16.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]': finished 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: osdmap e720: 8 total, 8 up, 8 in 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]': finished 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: osdmap e720: 8 total, 8 up, 8 in 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: pgmap v1118: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi045-74757-146", "mode": "writeback"}]': finished 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: osdmap e720: 8 total, 8 up, 8 in 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:16 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]: dispatch 2024-04-14T15:26:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:26:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[37795]: osdmap e721: 8 total, 8 up, 8 in 2024-04-14T15:26:17.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[37795]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[29624]: osdmap e721: 8 total, 8 up, 8 in 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:17 smithi045 ceph-mon[29624]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:17 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_count","val": "2"}]': finished 2024-04-14T15:26:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:17 smithi071 ceph-mon[33150]: osdmap e721: 8 total, 8 up, 8 in 2024-04-14T15:26:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:17 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:17 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]: dispatch 2024-04-14T15:26:17.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:17 smithi071 ceph-mon[33150]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 753 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-14T15:26:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:26:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[29624]: osdmap e722: 8 total, 8 up, 8 in 2024-04-14T15:26:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:18.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:18.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:26:18.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[37795]: osdmap e722: 8 total, 8 up, 8 in 2024-04-14T15:26:18.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:18.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:18 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:18 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_period","val": "600"}]': finished 2024-04-14T15:26:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:18 smithi071 ceph-mon[33150]: osdmap e722: 8 total, 8 up, 8 in 2024-04-14T15:26:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:18 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:18 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-04-14T15:26:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:26:19.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: osdmap e723: 8 total, 8 up, 8 in 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[37795]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: osdmap e723: 8 total, 8 up, 8 in 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:19 smithi045 ceph-mon[29624]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "hit_set_type","val": "bloom"}]': finished 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: osdmap e723: 8 total, 8 up, 8 in 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-04-14T15:26:19.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:19 smithi071 ceph-mon[33150]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: osdmap e724: 8 total, 8 up, 8 in 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[37795]: osdmap e725: 8 total, 8 up, 8 in 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: osdmap e724: 8 total, 8 up, 8 in 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:26:20.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:20 smithi045 ceph-mon[29624]: osdmap e725: 8 total, 8 up, 8 in 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: osdmap e724: 8 total, 8 up, 8 in 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]: dispatch 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-146","var": "target_max_objects","val": "1"}]': finished 2024-04-14T15:26:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:20 smithi071 ceph-mon[33150]: osdmap e725: 8 total, 8 up, 8 in 2024-04-14T15:26:21.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:21 smithi045 ceph-mon[37795]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:21.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:21 smithi045 ceph-mon[29624]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:21.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:21 smithi071 ceph-mon[33150]: pgmap v1127: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:22.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:22 smithi045 ceph-mon[37795]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:26:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:22 smithi045 ceph-mon[29624]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:26:22.758 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:22 smithi071 ceph-mon[33150]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-04-14T15:26:23.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:23 smithi045 ceph-mon[37795]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:26:23.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:23 smithi045 ceph-mon[29624]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:26:23.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:23 smithi071 ceph-mon[33150]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-14T15:26:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:24.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:25 smithi045 ceph-mon[37795]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:25.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:25 smithi045 ceph-mon[29624]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:25.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:25 smithi071 ceph-mon[33150]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-14T15:26:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:27.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:27 smithi045 ceph-mon[37795]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:26:27.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:27 smithi045 ceph-mon[29624]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:26:28.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:27 smithi071 ceph-mon[33150]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-04-14T15:26:28.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:26:28.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:28.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:28.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[29624]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:29.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:26:29.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:26:29.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:29.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[37795]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:29.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:26:29.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:26:29.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:29 smithi071 ceph-mon[33150]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:26:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:26:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:30.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:30 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:30.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:30 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:30.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:30 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:30.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:30 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:30 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:30 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:31.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[29624]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[29624]: osdmap e726: 8 total, 8 up, 8 in 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[37795]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:31.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[37795]: osdmap e726: 8 total, 8 up, 8 in 2024-04-14T15:26:31.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:31.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:31 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:31 smithi071 ceph-mon[33150]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 B/s wr, 1 op/s 2024-04-14T15:26:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:31 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:31 smithi071 ceph-mon[33150]: osdmap e726: 8 total, 8 up, 8 in 2024-04-14T15:26:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:31 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:32.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: osdmap e727: 8 total, 8 up, 8 in 2024-04-14T15:26:32.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:32.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: osdmap e727: 8 total, 8 up, 8 in 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:32.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:32 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]': finished 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: osdmap e727: 8 total, 8 up, 8 in 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:33.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:32 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-146"}]: dispatch 2024-04-14T15:26:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[29624]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:26:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[29624]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[29624]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:26:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[29624]: osdmap e728: 8 total, 8 up, 8 in 2024-04-14T15:26:33.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[37795]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:26:33.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[37795]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:33.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[37795]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:26:33.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:33 smithi045 ceph-mon[37795]: osdmap e728: 8 total, 8 up, 8 in 2024-04-14T15:26:34.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:33 smithi071 ceph-mon[33150]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:26:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:33 smithi071 ceph-mon[33150]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:33 smithi071 ceph-mon[33150]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-04-14T15:26:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:33 smithi071 ceph-mon[33150]: osdmap e728: 8 total, 8 up, 8 in 2024-04-14T15:26:34.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:34.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[29624]: osdmap e729: 8 total, 8 up, 8 in 2024-04-14T15:26:34.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:34.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:34.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:34.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[37795]: osdmap e729: 8 total, 8 up, 8 in 2024-04-14T15:26:34.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:34.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:34.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:35.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:34 smithi071 ceph-mon[33150]: osdmap e729: 8 total, 8 up, 8 in 2024-04-14T15:26:35.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:34 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:35.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:34 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:35.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:35.609 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=sqlstore.transactions t=2024-04-14T15:26:35.213958702Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-04-14T15:26:35.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[37795]: pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:35.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:35.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[37795]: osdmap e730: 8 total, 8 up, 8 in 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[29624]: pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[29624]: osdmap e730: 8 total, 8 up, 8 in 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:35.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:35 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-mon[33150]: pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-mon[33150]: osdmap e730: 8 total, 8 up, 8 in 2024-04-14T15:26:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:35 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-04-14T15:26:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[37795]: osdmap e731: 8 total, 8 up, 8 in 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[29624]: osdmap e731: 8 total, 8 up, 8 in 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:36.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:36 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:37.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:36 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148", "force_nonempty": "--force-nonempty" }]': finished 2024-04-14T15:26:37.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:36 smithi071 ceph-mon[33150]: osdmap e731: 8 total, 8 up, 8 in 2024-04-14T15:26:37.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:36 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:37.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:36 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:38.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]': finished 2024-04-14T15:26:38.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: osdmap e732: 8 total, 8 up, 8 in 2024-04-14T15:26:38.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:37 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]': finished 2024-04-14T15:26:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: osdmap e732: 8 total, 8 up, 8 in 2024-04-14T15:26:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]': finished 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: osdmap e732: 8 total, 8 up, 8 in 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:38.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:37 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-148"}]: dispatch 2024-04-14T15:26:39.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:38 smithi071 ceph-mon[33150]: osdmap e733: 8 total, 8 up, 8 in 2024-04-14T15:26:39.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:38 smithi071 ceph-mon[33150]: osdmap e734: 8 total, 8 up, 8 in 2024-04-14T15:26:39.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:38 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:39.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:38 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[37795]: osdmap e733: 8 total, 8 up, 8 in 2024-04-14T15:26:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[37795]: osdmap e734: 8 total, 8 up, 8 in 2024-04-14T15:26:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:39.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[29624]: osdmap e733: 8 total, 8 up, 8 in 2024-04-14T15:26:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[29624]: osdmap e734: 8 total, 8 up, 8 in 2024-04-14T15:26:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:39.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:38 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:40.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:39 smithi071 ceph-mon[33150]: pgmap v1144: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:40.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:39 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:40.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:39 smithi071 ceph-mon[33150]: osdmap e735: 8 total, 8 up, 8 in 2024-04-14T15:26:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[37795]: pgmap v1144: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[37795]: osdmap e735: 8 total, 8 up, 8 in 2024-04-14T15:26:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[29624]: pgmap v1144: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:40.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:39 smithi045 ceph-mon[29624]: osdmap e735: 8 total, 8 up, 8 in 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:41.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:40 smithi071 ceph-mon[33150]: osdmap e736: 8 total, 8 up, 8 in 2024-04-14T15:26:41.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[37795]: osdmap e736: 8 total, 8 up, 8 in 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-150"}]: dispatch 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:41.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:40 smithi045 ceph-mon[29624]: osdmap e736: 8 total, 8 up, 8 in 2024-04-14T15:26:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[37795]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[37795]: osdmap e737: 8 total, 8 up, 8 in 2024-04-14T15:26:42.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:42.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[29624]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[29624]: osdmap e737: 8 total, 8 up, 8 in 2024-04-14T15:26:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:42.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:42 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:42 smithi071 ceph-mon[33150]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-04-14T15:26:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:42 smithi071 ceph-mon[33150]: osdmap e737: 8 total, 8 up, 8 in 2024-04-14T15:26:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:42 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:42 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: pgmap v1150: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: osdmap e738: 8 total, 8 up, 8 in 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:44.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: pgmap v1150: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:44.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: osdmap e738: 8 total, 8 up, 8 in 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: pgmap v1150: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: osdmap e738: 8 total, 8 up, 8 in 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:44.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-152"}]: dispatch 2024-04-14T15:26:44.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:26:44.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:44.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:45.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:44 smithi071 ceph-mon[33150]: osdmap e739: 8 total, 8 up, 8 in 2024-04-14T15:26:45.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:45.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:44 smithi045 ceph-mon[37795]: osdmap e739: 8 total, 8 up, 8 in 2024-04-14T15:26:45.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:45.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:44 smithi045 ceph-mon[29624]: osdmap e739: 8 total, 8 up, 8 in 2024-04-14T15:26:45.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:45 smithi071 ceph-mon[33150]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:45 smithi071 ceph-mon[33150]: osdmap e740: 8 total, 8 up, 8 in 2024-04-14T15:26:46.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:45 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.108 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:45 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[29624]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[29624]: osdmap e740: 8 total, 8 up, 8 in 2024-04-14T15:26:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[37795]: pgmap v1153: 172 pgs: 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:46.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[37795]: osdmap e740: 8 total, 8 up, 8 in 2024-04-14T15:26:46.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.249 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:45 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-04-14T15:26:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:46.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:46.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: osdmap e741: 8 total, 8 up, 8 in 2024-04-14T15:26:46.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: osdmap e741: 8 total, 8 up, 8 in 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:46.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:46.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:46.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:46 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi045-74757-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: osdmap e741: 8 total, 8 up, 8 in 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi045-74757-111","var": "dedup_tier","val": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:47.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:46 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi045-74757-111", "tierpool": "test-rados-api-smithi045-74757-154"}]: dispatch 2024-04-14T15:26:48.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:47 smithi071 ceph-mon[33150]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:48.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:47 smithi071 ceph-mon[33150]: osdmap e742: 8 total, 8 up, 8 in 2024-04-14T15:26:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:47 smithi045 ceph-mon[29624]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:47 smithi045 ceph-mon[29624]: osdmap e742: 8 total, 8 up, 8 in 2024-04-14T15:26:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:47 smithi045 ceph-mon[37795]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-14T15:26:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:47 smithi045 ceph-mon[37795]: osdmap e742: 8 total, 8 up, 8 in 2024-04-14T15:26:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:48 smithi071 ceph-mon[33150]: osdmap e743: 8 total, 8 up, 8 in 2024-04-14T15:26:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:48 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[29624]: osdmap e743: 8 total, 8 up, 8 in 2024-04-14T15:26:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[37795]: osdmap e743: 8 total, 8 up, 8 in 2024-04-14T15:26:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:48 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:49.806 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (8057 ms) 2024-04-14T15:26:49.806 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13072 ms) 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8039 ms) 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13063 ms) 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-04-14T15:26:49.807 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7781 ms) 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8046 ms) 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: hmm, no HitSet yet 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: ok, hit_set contains 337:602f83fe:::foo:head 2024-04-14T15:26:49.808 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9066 ms) 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,0 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: first is 1713108325 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,0 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,0 2024-04-14T15:26:49.809 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,0 2024-04-14T15:26:49.810 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,0 2024-04-14T15:26:49.810 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,0 2024-04-14T15:26:49.810 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,0 2024-04-14T15:26:49.810 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,0 2024-04-14T15:26:49.810 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,1713108333,1713108334,0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,1713108333,1713108334,0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108325,1713108327,1713108328,1713108330,1713108331,1713108333,1713108334,0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: got ls 1713108328,1713108330,1713108331,1713108333,1713108334,1713108336,1713108337,0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: first now 1713108328, trimmed 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21087 ms) 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: foo0 2024-04-14T15:26:49.811 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14070 ms) 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17941 ms) 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22378 ms) 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5019 ms) 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-04-14T15:26:49.812 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3007 ms) 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3011 ms) 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3023 ms) 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232242 ms total) 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (846277 ms total) 2024-04-14T15:26:49.813 INFO:tasks.workunit.client.0.smithi045.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-04-14T15:26:49.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74444 2024-04-14T15:26:49.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74444 2024-04-14T15:26:49.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.820 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74649 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74649 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75246 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75246 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74995 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74995 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74764 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74764 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74531 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74531 2024-04-14T15:26:49.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74836 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74836 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75506 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75506 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75542 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75542 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74547 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74547 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.822 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75272 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75272 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75138 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75138 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74419 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74419 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74481 2024-04-14T15:26:49.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74481 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75177 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75177 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74410 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74410 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74431 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74431 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75684 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75684 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74703 2024-04-14T15:26:49.824 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74703 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75764 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75764 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74810 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74810 2024-04-14T15:26:49.825 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75402 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75402 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75801 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75801 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74733 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74733 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75227 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75227 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.826 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75620 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75620 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75327 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75327 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75738 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75738 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75115 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75115 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74602 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74602 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.827 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75786 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75786 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75025 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75025 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74403 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74403 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74461 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74461 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74911 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74911 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.828 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=74500 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 74500 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75053 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75053 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75065 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75065 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75588 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75588 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ for t in "${!pids[@]}" 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=75829 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 75829 2024-04-14T15:26:49.829 INFO:tasks.workunit.client.0.smithi045.stderr:+ exit 0 2024-04-14T15:26:49.830 INFO:tasks.workunit.client.0.smithi045.stderr:+ cleanup 2024-04-14T15:26:49.830 INFO:tasks.workunit.client.0.smithi045.stderr:+ pkill -P 74397 2024-04-14T15:26:49.831 INFO:tasks.workunit.client.0.smithi045.stderr:+ true 2024-04-14T15:26:49.831 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-14T15:26:49.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-14T15:26:49.866 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-04-14T15:26:49.867 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-04-14T15:26:49.867 DEBUG:teuthology.orchestra.run.smithi045:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=05c9d46e7de5db2cb434c177a6bbfc56f472d09b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-04-14T15:26:49.926 INFO:tasks.workunit.client.0.smithi045.stderr:++ uuidgen 2024-04-14T15:26:49.933 INFO:tasks.workunit.client.0.smithi045.stderr:+ p=c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:26:49.933 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool create c485d52f-0910-43c8-a4a6-e912c4af97a1 12 2024-04-14T15:26:50.020 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.019+0000 7fd8b5155640 1 -- 172.21.15.45:0/540975810 >> v1:172.21.15.45:6789/0 conn(0x7fd8b01020b0 legacy=0x7fd8b0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:50.021 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.019+0000 7fd8b5155640 1 -- 172.21.15.45:0/540975810 shutdown_connections 2024-04-14T15:26:50.021 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.019+0000 7fd8b5155640 1 -- 172.21.15.45:0/540975810 >> 172.21.15.45:0/540975810 conn(0x7fd8b00fd930 msgr2=0x7fd8b00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:50.021 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.019+0000 7fd8b5155640 1 -- 172.21.15.45:0/540975810 shutdown_connections 2024-04-14T15:26:50.021 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.019+0000 7fd8b5155640 1 -- 172.21.15.45:0/540975810 wait complete. 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.020+0000 7fd8b5155640 1 Processor -- start 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.020+0000 7fd8b5155640 1 -- start start 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.020+0000 7fd8b5155640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8b01a9110 con 0x7fd8b01020b0 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.020+0000 7fd8b5155640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8b01aa350 con 0x7fd8b010e8a0 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.020+0000 7fd8b5155640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8b01ab590 con 0x7fd8b010ad30 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd8ae59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd8b010ad30 0x7fd8b01a4100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:48178/0 (socket says 172.21.15.45:48178) 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd8ae59c640 1 -- 172.21.15.45:0/3133530319 learned_addr learned my addr 172.21.15.45:0/3133530319 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd8aed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd8b01020b0 0x7fd8b01073c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:55924/0 (socket says 172.21.15.45:55924) 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643473844 0 0) 0x7fd8b01a9110 con 0x7fd8b01020b0 2024-04-14T15:26:50.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd898003520 con 0x7fd8b01020b0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900689767 0 0) 0x7fd8b01ab590 con 0x7fd8b010ad30 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8b01a9110 con 0x7fd8b010ad30 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374890376 0 0) 0x7fd8b01aa350 con 0x7fd8b010e8a0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8b01ab590 con 0x7fd8b010e8a0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2156458420 0 0) 0x7fd898003520 con 0x7fd8b01020b0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8b01aa350 con 0x7fd8b01020b0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1149786884 0 0) 0x7fd8b01a9110 con 0x7fd8b010ad30 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd898003520 con 0x7fd8b010ad30 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8a0004530 con 0x7fd8b01020b0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8a4001990 con 0x7fd8b010ad30 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2516943716 0 0) 0x7fd8b01ab590 con 0x7fd8b010e8a0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8b01a9110 con 0x7fd8b010e8a0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8a8001bb0 con 0x7fd8b010e8a0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 23591416 0 0) 0x7fd8b01aa350 con 0x7fd8b01020b0 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 >> v1:172.21.15.45:6790/0 conn(0x7fd8b010ad30 legacy=0x7fd8b01a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:50.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 >> v1:172.21.15.71:6789/0 conn(0x7fd8b010e8a0 legacy=0x7fd8b01a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b01ac7d0 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b01ab7c0 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.021+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd8b01abc90 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.022+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd8a00011a0 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.022+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd8a0004c70 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.022+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c005080 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.023+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fd8a00035f0 con 0x7fd8b01020b0 2024-04-14T15:26:50.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.024+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(745..745 src has 1..745) ==== 7398+0+0 (unknown 2936269545 0 0) 0x7fd8a008a4c0 con 0x7fd8b01020b0 2024-04-14T15:26:50.029 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.029+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fd8a0057310 con 0x7fd8b01020b0 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: pgmap v1159: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: osdmap e744: 8 total, 8 up, 8 in 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:49 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.216 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.216+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12} v 0) -- 0x7fd89c005370 con 0x7fd8b01020b0 2024-04-14T15:26:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: pgmap v1159: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: osdmap e744: 8 total, 8 up, 8 in 2024-04-14T15:26:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: pgmap v1159: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: osdmap e744: 8 total, 8 up, 8 in 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3174251209' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:49 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]: dispatch 2024-04-14T15:26:50.806 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.805+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]=0 pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' created v746) ==== 176+0+0 (unknown 2652553383 0 0) 0x7fd8a005afc0 con 0x7fd8b01020b0 2024-04-14T15:26:50.925 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.924+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12} v 0) -- 0x7fd89c001170 con 0x7fd8b01020b0 2024-04-14T15:26:50.926 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.925+0000 7fd897fff640 1 -- 172.21.15.45:0/3133530319 <== mon.0 v1:172.21.15.45:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]=0 pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' already exists v746) ==== 183+0+0 (unknown 1812898697 0 0) 0x7fd89c001170 con 0x7fd8b01020b0 2024-04-14T15:26:50.926 INFO:tasks.workunit.client.0.smithi045.stderr:pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' already exists 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 >> v1:172.21.15.45:6800/1718216366 conn(0x7fd89806f930 legacy=0x7fd898071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 >> v1:172.21.15.45:6789/0 conn(0x7fd8b01020b0 legacy=0x7fd8b01073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 shutdown_connections 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 >> 172.21.15.45:0/3133530319 conn(0x7fd8b00fd930 msgr2=0x7fd8b0107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 shutdown_connections 2024-04-14T15:26:50.928 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:50.927+0000 7fd8b5155640 1 -- 172.21.15.45:0/3133530319 wait complete. 2024-04-14T15:26:50.936 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_objects 10 2024-04-14T15:26:51.023 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- 172.21.15.45:0/3650370547 >> v1:172.21.15.45:6789/0 conn(0x7f88bc10ad30 legacy=0x7f88bc10d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:51.023 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- 172.21.15.45:0/3650370547 shutdown_connections 2024-04-14T15:26:51.023 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- 172.21.15.45:0/3650370547 >> 172.21.15.45:0/3650370547 conn(0x7f88bc0fd950 msgr2=0x7f88bc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:51.023 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- 172.21.15.45:0/3650370547 shutdown_connections 2024-04-14T15:26:51.023 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- 172.21.15.45:0/3650370547 wait complete. 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 Processor -- start 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- start start 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.020+0000 7f88c4660640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f88bc1a6f40 con 0x7f88bc10ad30 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c4660640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f88bc1a8180 con 0x7f88bc10e8a0 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c4660640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f88bc1a93c0 con 0x7f88bc1020b0 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c23fc640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f88bc1020b0 0x7f88bc107450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:59578/0 (socket says 172.21.15.45:59578) 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c23fc640 1 -- 172.21.15.45:0/2212721452 learned_addr learned my addr 172.21.15.45:0/2212721452 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2912190496 0 0) 0x7f88bc1a93c0 con 0x7f88bc1020b0 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88a8003520 con 0x7f88bc1020b0 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887317210 0 0) 0x7f88bc1a6f40 con 0x7f88bc10ad30 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88bc1a93c0 con 0x7f88bc10ad30 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 451329860 0 0) 0x7f88bc1a8180 con 0x7f88bc10e8a0 2024-04-14T15:26:51.024 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88bc1a6f40 con 0x7f88bc10e8a0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1402775280 0 0) 0x7f88a8003520 con 0x7f88bc1020b0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88bc1a8180 con 0x7f88bc1020b0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f88b8001930 con 0x7f88bc1020b0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 540534053 0 0) 0x7f88bc1a93c0 con 0x7f88bc10ad30 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88a8003520 con 0x7f88bc10ad30 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f88ac004410 con 0x7f88bc10ad30 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4100038438 0 0) 0x7f88bc1a6f40 con 0x7f88bc10e8a0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88bc1a93c0 con 0x7f88bc10e8a0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f88b4001470 con 0x7f88bc10e8a0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 654112929 0 0) 0x7f88bc1a8180 con 0x7f88bc1020b0 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 >> v1:172.21.15.71:6789/0 conn(0x7f88bc10e8a0 legacy=0x7f88bc1a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:51.025 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 >> v1:172.21.15.45:6789/0 conn(0x7f88bc10ad30 legacy=0x7f88bc1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1aa600 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc1a7170 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a7640 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f88b8001d00 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.021+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f88b8005c90 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.022+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a0005080 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.023+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f88b80038c0 con 0x7f88bc1020b0 2024-04-14T15:26:51.026 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.023+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(746..746 src has 1..746) ==== 7773+0+0 (unknown 898294301 0 0) 0x7f88b808a920 con 0x7f88bc1020b0 2024-04-14T15:26:51.029 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.028+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f88b80574c0 con 0x7f88bc1020b0 2024-04-14T15:26:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:50 smithi071 ceph-mon[33150]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:50 smithi071 ceph-mon[33150]: osdmap e745: 8 total, 8 up, 8 in 2024-04-14T15:26:51.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:50 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:51.216 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.215+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"} v 0) -- 0x7f88a0005370 con 0x7f88bc1020b0 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[29624]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[29624]: osdmap e745: 8 total, 8 up, 8 in 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[37795]: from='client.56888 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi045-74757-111"}]': finished 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[37795]: osdmap e745: 8 total, 8 up, 8 in 2024-04-14T15:26:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:50 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:51.812 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.811+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v747) ==== 223+0+0 (unknown 2595265389 0 0) 0x7f88b805b170 con 0x7f88bc1020b0 2024-04-14T15:26:51.932 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:51.932+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"} v 0) -- 0x7f88a0001090 con 0x7f88bc1020b0 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]': finished 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: osdmap e746: 8 total, 8 up, 8 in 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:51 smithi071 ceph-mon[33150]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:52.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]': finished 2024-04-14T15:26:52.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: osdmap e746: 8 total, 8 up, 8 in 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[37795]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]': finished 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: osdmap e746: 8 total, 8 up, 8 in 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3133530319' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pg_num": 12}]: dispatch 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:51 smithi045 ceph-mon[29624]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:52.827 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.825+0000 7f88b37fe640 1 -- 172.21.15.45:0/2212721452 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v748) ==== 223+0+0 (unknown 1636079564 0 0) 0x7f88b805fa30 con 0x7f88bc1020b0 2024-04-14T15:26:52.827 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_objects = 10 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 >> v1:172.21.15.45:6800/1718216366 conn(0x7f88a806f950 legacy=0x7f88a8071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 >> v1:172.21.15.45:6790/0 conn(0x7f88bc1020b0 legacy=0x7f88bc107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 shutdown_connections 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 >> 172.21.15.45:0/2212721452 conn(0x7f88bc0fd950 msgr2=0x7f88bc1119f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 shutdown_connections 2024-04-14T15:26:52.830 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.829+0000 7f88c4660640 1 -- 172.21.15.45:0/2212721452 wait complete. 2024-04-14T15:26:52.841 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool application enable c485d52f-0910-43c8-a4a6-e912c4af97a1 rados 2024-04-14T15:26:52.930 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.929+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3346746596 >> v1:172.21.15.45:6790/0 conn(0x7f99ec10ae10 legacy=0x7f99ec10d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:52.930 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.929+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3346746596 shutdown_connections 2024-04-14T15:26:52.930 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.929+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3346746596 >> 172.21.15.45:0/3346746596 conn(0x7f99ec0fc850 msgr2=0x7f99ec0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:52.930 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.929+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3346746596 shutdown_connections 2024-04-14T15:26:52.931 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.929+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3346746596 wait complete. 2024-04-14T15:26:52.933 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99f0cc0640 1 Processor -- start 2024-04-14T15:26:52.933 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99f0cc0640 1 -- start start 2024-04-14T15:26:52.933 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99f0cc0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f99ec1a6e60 con 0x7f99ec10ae10 2024-04-14T15:26:52.933 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99f0cc0640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f99ec1a80a0 con 0x7f99ec10e920 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99f0cc0640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f99ec1a92e0 con 0x7f99ec103010 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99ea59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f99ec103010 0x7f99ec10a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:59598/0 (socket says 172.21.15.45:59598) 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99e9d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f99ec10ae10 0x7f99ec1a1d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53128/0 (socket says 172.21.15.45:53128) 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99ea59c640 1 -- 172.21.15.45:0/3573524803 learned_addr learned my addr 172.21.15.45:0/3573524803 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2314294294 0 0) 0x7f99ec1a6e60 con 0x7f99ec10ae10 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99d4003160 con 0x7f99ec10ae10 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873053339 0 0) 0x7f99ec1a92e0 con 0x7f99ec103010 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.930+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ec1a6e60 con 0x7f99ec103010 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522529888 0 0) 0x7f99ec1a80a0 con 0x7f99ec10e920 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ec1a92e0 con 0x7f99ec10e920 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1083550581 0 0) 0x7f99d4003160 con 0x7f99ec10ae10 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f99ec1a80a0 con 0x7f99ec10ae10 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f99e00041e0 con 0x7f99ec10ae10 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1243018410 0 0) 0x7f99ec1a6e60 con 0x7f99ec103010 2024-04-14T15:26:52.934 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f99d4003160 con 0x7f99ec103010 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f99dc001bf0 con 0x7f99ec103010 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1424988319 0 0) 0x7f99ec1a92e0 con 0x7f99ec10e920 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f99ec1a6e60 con 0x7f99ec10e920 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f99e4001890 con 0x7f99ec10e920 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2106647410 0 0) 0x7f99ec1a80a0 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 >> v1:172.21.15.45:6790/0 conn(0x7f99ec103010 legacy=0x7f99ec10a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 >> v1:172.21.15.71:6789/0 conn(0x7f99ec10e920 legacy=0x7f99ec1a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec1aa520 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec1a7090 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f99ec1a7560 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f99e00019e0 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.931+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f99e0004be0 con 0x7f99ec10ae10 2024-04-14T15:26:52.935 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.932+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99cc005080 con 0x7f99ec10ae10 2024-04-14T15:26:52.936 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.933+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f99e0004e60 con 0x7f99ec10ae10 2024-04-14T15:26:52.936 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.933+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(748..748 src has 1..748) ==== 7773+0+0 (unknown 2663511919 0 0) 0x7f99e008a790 con 0x7f99ec10ae10 2024-04-14T15:26:52.939 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:52.938+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f99e0057330 con 0x7f99ec10ae10 2024-04-14T15:26:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:52 smithi071 ceph-mon[33150]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:52 smithi071 ceph-mon[33150]: osdmap e747: 8 total, 8 up, 8 in 2024-04-14T15:26:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:52 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:52 smithi071 ceph-mon[33150]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.128 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:53.128+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"} v 0) -- 0x7f99cc005370 con 0x7f99ec10ae10 2024-04-14T15:26:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[29624]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[29624]: osdmap e747: 8 total, 8 up, 8 in 2024-04-14T15:26:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[29624]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[37795]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[37795]: osdmap e747: 8 total, 8 up, 8 in 2024-04-14T15:26:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2212721452' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:52 smithi045 ceph-mon[37795]: from='client.56900 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:26:53.826 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:53.822+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]=0 enabled application 'rados' on pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' v749) ==== 213+0+0 (unknown 3815525274 0 0) 0x7f99e005afe0 con 0x7f99ec10ae10 2024-04-14T15:26:53.941 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:53.940+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"} v 0) -- 0x7f99cc001090 con 0x7f99ec10ae10 2024-04-14T15:26:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:53 smithi071 ceph-mon[33150]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:53 smithi071 ceph-mon[33150]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:53 smithi071 ceph-mon[33150]: osdmap e748: 8 total, 8 up, 8 in 2024-04-14T15:26:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:53 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[29624]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[29624]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[29624]: osdmap e748: 8 total, 8 up, 8 in 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[37795]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[37795]: from='client.56900 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[37795]: osdmap e748: 8 total, 8 up, 8 in 2024-04-14T15:26:54.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:53 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:54.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:26:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:26:54.834 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.833+0000 7f99db7fe640 1 -- 172.21.15.45:0/3573524803 <== mon.0 v1:172.21.15.45:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]=0 enabled application 'rados' on pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' v750) ==== 213+0+0 (unknown 2903396528 0 0) 0x7f99e005f8a0 con 0x7f99ec10ae10 2024-04-14T15:26:54.835 INFO:tasks.workunit.client.0.smithi045.stderr:enabled application 'rados' on pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' 2024-04-14T15:26:54.838 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.837+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 >> v1:172.21.15.45:6800/1718216366 conn(0x7f99d406f950 legacy=0x7f99d4071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:54.838 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.837+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 >> v1:172.21.15.45:6789/0 conn(0x7f99ec10ae10 legacy=0x7f99ec1a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:26:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.837+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 shutdown_connections 2024-04-14T15:26:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.838+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 >> 172.21.15.45:0/3573524803 conn(0x7f99ec0fc850 msgr2=0x7f99ec110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:26:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.838+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 shutdown_connections 2024-04-14T15:26:54.839 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:26:54.838+0000 7f99f0cc0640 1 -- 172.21.15.45:0/3573524803 wait complete. 2024-04-14T15:26:54.851 INFO:tasks.workunit.client.0.smithi045.stderr:++ seq 1 10 2024-04-14T15:26:54.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:54.858 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj1 /etc/passwd 2024-04-14T15:26:54.894 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:54.894 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj2 /etc/passwd 2024-04-14T15:26:54.930 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:54.930 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj3 /etc/passwd 2024-04-14T15:26:54.964 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:54.964 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj4 /etc/passwd 2024-04-14T15:26:54.998 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:54.998 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj5 /etc/passwd 2024-04-14T15:26:55.030 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:55.030 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj6 /etc/passwd 2024-04-14T15:26:55.063 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:55.063 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj7 /etc/passwd 2024-04-14T15:26:55.095 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:55.095 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj8 /etc/passwd 2024-04-14T15:26:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:54 smithi071 ceph-mon[33150]: osdmap e749: 8 total, 8 up, 8 in 2024-04-14T15:26:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:54 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:55.128 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:55.128 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj9 /etc/passwd 2024-04-14T15:26:55.161 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:26:55.161 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put obj10 /etc/passwd 2024-04-14T15:26:55.193 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 30 2024-04-14T15:26:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[37795]: osdmap e749: 8 total, 8 up, 8 in 2024-04-14T15:26:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[29624]: osdmap e749: 8 total, 8 up, 8 in 2024-04-14T15:26:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]: dispatch 2024-04-14T15:26:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:26:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:55 smithi071 ceph-mon[33150]: pgmap v1168: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:26:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:55 smithi071 ceph-mon[33150]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:55 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:55 smithi071 ceph-mon[33150]: osdmap e750: 8 total, 8 up, 8 in 2024-04-14T15:26:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[37795]: pgmap v1168: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[37795]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[37795]: osdmap e750: 8 total, 8 up, 8 in 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[29624]: pgmap v1168: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[29624]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3573524803' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "app": "rados"}]': finished 2024-04-14T15:26:56.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:55 smithi045 ceph-mon[29624]: osdmap e750: 8 total, 8 up, 8 in 2024-04-14T15:26:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:26:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:26:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:26:58.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:57 smithi071 ceph-mon[33150]: pgmap v1170: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:26:58.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:57 smithi045 ceph-mon[29624]: pgmap v1170: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:26:58.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:57 smithi045 ceph-mon[37795]: pgmap v1170: 176 pgs: 176 active+clean; 582 KiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:26:59.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:58 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:59.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:58 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:26:59.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:58 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:00.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:26:59 smithi071 ceph-mon[33150]: pgmap v1171: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-04-14T15:27:00.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:26:59 smithi045 ceph-mon[37795]: pgmap v1171: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-04-14T15:27:00.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:26:59 smithi045 ceph-mon[29624]: pgmap v1171: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-04-14T15:27:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:00 smithi071 ceph-mon[33150]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_objects: 10) 2024-04-14T15:27:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:00 smithi071 ceph-mon[33150]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:01.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:00 smithi071 ceph-mon[33150]: osdmap e751: 8 total, 8 up, 8 in 2024-04-14T15:27:01.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[29624]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_objects: 10) 2024-04-14T15:27:01.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[29624]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:01.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[29624]: osdmap e751: 8 total, 8 up, 8 in 2024-04-14T15:27:01.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[37795]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_objects: 10) 2024-04-14T15:27:01.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[37795]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:01.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:00 smithi045 ceph-mon[37795]: osdmap e751: 8 total, 8 up, 8 in 2024-04-14T15:27:02.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:02 smithi045 ceph-mon[29624]: pgmap v1173: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 18 KiB/s wr, 2 op/s 2024-04-14T15:27:02.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:02 smithi045 ceph-mon[37795]: pgmap v1173: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 18 KiB/s wr, 2 op/s 2024-04-14T15:27:02.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:02 smithi071 ceph-mon[33150]: pgmap v1173: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 18 KiB/s wr, 2 op/s 2024-04-14T15:27:04.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:04 smithi045 ceph-mon[37795]: pgmap v1174: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-04-14T15:27:04.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:04 smithi045 ceph-mon[29624]: pgmap v1174: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-04-14T15:27:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:04 smithi071 ceph-mon[33150]: pgmap v1174: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-04-14T15:27:04.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:05.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:05.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:06.499 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:06 smithi045 ceph-mon[37795]: pgmap v1175: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-14T15:27:06.500 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:06 smithi045 ceph-mon[29624]: pgmap v1175: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-14T15:27:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:06 smithi071 ceph-mon[33150]: pgmap v1175: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-04-14T15:27:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:08.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:08 smithi045 ceph-mon[37795]: pgmap v1176: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:27:08.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:08 smithi045 ceph-mon[29624]: pgmap v1176: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:27:08.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:08 smithi071 ceph-mon[33150]: pgmap v1176: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:27:10.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:10 smithi045 ceph-mon[37795]: pgmap v1177: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:10.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:10 smithi045 ceph-mon[29624]: pgmap v1177: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:10 smithi071 ceph-mon[33150]: pgmap v1177: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:12.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:12 smithi045 ceph-mon[37795]: pgmap v1178: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-04-14T15:27:12.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:12 smithi045 ceph-mon[29624]: pgmap v1178: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-04-14T15:27:12.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:12 smithi071 ceph-mon[33150]: pgmap v1178: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-04-14T15:27:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[29624]: pgmap v1179: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:14.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[37795]: pgmap v1179: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:14.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:14 smithi071 ceph-mon[33150]: pgmap v1179: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:14.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:15.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:15.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:15.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:16.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:16 smithi045 ceph-mon[37795]: pgmap v1180: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:16.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:16 smithi045 ceph-mon[29624]: pgmap v1180: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:16.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:16 smithi071 ceph-mon[33150]: pgmap v1180: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:18.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:18 smithi045 ceph-mon[37795]: pgmap v1181: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:18.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:18 smithi045 ceph-mon[29624]: pgmap v1181: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:18.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:18 smithi071 ceph-mon[33150]: pgmap v1181: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:20.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:20 smithi045 ceph-mon[37795]: pgmap v1182: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:20.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:20 smithi045 ceph-mon[29624]: pgmap v1182: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:20.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:20 smithi071 ceph-mon[33150]: pgmap v1182: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:22.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:22 smithi045 ceph-mon[29624]: pgmap v1183: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:22.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:22 smithi045 ceph-mon[37795]: pgmap v1183: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:22.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:22 smithi071 ceph-mon[33150]: pgmap v1183: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:24.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:24 smithi045 ceph-mon[37795]: pgmap v1184: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:24.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:24 smithi045 ceph-mon[29624]: pgmap v1184: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:24 smithi071 ceph-mon[33150]: pgmap v1184: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:24.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:25.195 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=145360 2024-04-14T15:27:25.195 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_objects 100 2024-04-14T15:27:25.195 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put onemore /etc/passwd 2024-04-14T15:27:25.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 -- 172.21.15.45:0/835279866 >> v1:172.21.15.45:6789/0 conn(0x7f0610103010 legacy=0x7f06101033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:25.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 -- 172.21.15.45:0/835279866 shutdown_connections 2024-04-14T15:27:25.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 -- 172.21.15.45:0/835279866 >> 172.21.15.45:0/835279866 conn(0x7f06100fc850 msgr2=0x7f06100fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:27:25.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 -- 172.21.15.45:0/835279866 shutdown_connections 2024-04-14T15:27:25.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 -- 172.21.15.45:0/835279866 wait complete. 2024-04-14T15:27:25.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.295+0000 7f0615aa6640 1 Processor -- start 2024-04-14T15:27:25.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f0615aa6640 1 -- start start 2024-04-14T15:27:25.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f0615aa6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06101a6eb0 con 0x7f0610103010 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f0615aa6640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06101a80f0 con 0x7f061010ae10 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f0615aa6640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f06101a9330 con 0x7f061010e920 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0610103010 0x7f061010a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51144/0 (socket says 172.21.15.45:51144) 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060f7fe640 1 -- 172.21.15.45:0/4257852180 learned_addr learned my addr 172.21.15.45:0/4257852180 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58203778 0 0) 0x7f06101a9330 con 0x7f061010e920 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05f8003520 con 0x7f061010e920 2024-04-14T15:27:25.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388616712 0 0) 0x7f06101a6eb0 con 0x7f0610103010 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06101a9330 con 0x7f0610103010 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 680361752 0 0) 0x7f06101a80f0 con 0x7f061010ae10 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06101a6eb0 con 0x7f061010ae10 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3429763691 0 0) 0x7f05f8003520 con 0x7f061010e920 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06101a80f0 con 0x7f061010e920 2024-04-14T15:27:25.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0608001470 con 0x7f061010e920 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2831237097 0 0) 0x7f06101a9330 con 0x7f0610103010 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f05f8003520 con 0x7f0610103010 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0600000fe0 con 0x7f0610103010 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.296+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3553248366 0 0) 0x7f06101a6eb0 con 0x7f061010ae10 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06101a9330 con 0x7f061010ae10 2024-04-14T15:27:25.305 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0604001870 con 0x7f061010ae10 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1468841986 0 0) 0x7f06101a80f0 con 0x7f061010e920 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 >> v1:172.21.15.71:6789/0 conn(0x7f061010ae10 legacy=0x7f06101a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 >> v1:172.21.15.45:6789/0 conn(0x7f0610103010 legacy=0x7f061010a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06101aa570 con 0x7f061010e920 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f06101a70e0 con 0x7f061010e920 2024-04-14T15:27:25.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f06101a75b0 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f0608003e00 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.297+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f0608004c80 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.298+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc005080 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.298+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f060801b830 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.299+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(751..751 src has 1..751) ==== 7786+0+0 (unknown 864604719 0 0) 0x7f0608089870 con 0x7f061010e920 2024-04-14T15:27:25.307 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.299+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=752}) -- 0x7f06101a80f0 con 0x7f061010e920 2024-04-14T15:27:25.308 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.303+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f0608056540 con 0x7f061010e920 2024-04-14T15:27:25.495 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:25.495+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"} v 0) -- 0x7f05fc005370 con 0x7f061010e920 2024-04-14T15:27:25.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:25.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:25.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:26.209 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:26.209+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 10 ==== osd_map(752..752 src has 1..752) ==== 628+0+0 (unknown 1090787417 0 0) 0x7f0608052420 con 0x7f061010e920 2024-04-14T15:27:26.209 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:26.209+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=753}) -- 0x7f05f8003520 con 0x7f061010e920 2024-04-14T15:27:26.213 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:26.213+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v752) ==== 225+0+0 (unknown 1396918606 0 0) 0x7f060805a1f0 con 0x7f061010e920 2024-04-14T15:27:26.330 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:26.330+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"} v 0) -- 0x7f05fc005b80 con 0x7f061010e920 2024-04-14T15:27:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[29624]: pgmap v1185: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:26.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[29624]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[37795]: pgmap v1185: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-mon[37795]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:26 smithi071 ceph-mon[33150]: pgmap v1185: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:26 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:26 smithi071 ceph-mon[33150]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:27.223 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.223+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 12 ==== osd_map(753..753 src has 1..753) ==== 628+0+0 (unknown 4251179851 0 0) 0x7f06080557a0 con 0x7f061010e920 2024-04-14T15:27:27.223 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.223+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=754}) -- 0x7f06101a6eb0 con 0x7f061010e920 2024-04-14T15:27:27.228 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.227+0000 7f060cff9640 1 -- 172.21.15.45:0/4257852180 <== mon.2 v1:172.21.15.45:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v753) ==== 225+0+0 (unknown 2074113334 0 0) 0x7f060805eab0 con 0x7f061010e920 2024-04-14T15:27:27.228 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_objects = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 >> v1:172.21.15.45:6800/1718216366 conn(0x7f05f806f8d0 legacy=0x7f05f8071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 >> v1:172.21.15.45:6790/0 conn(0x7f061010e920 legacy=0x7f06101a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 shutdown_connections 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 >> 172.21.15.45:0/4257852180 conn(0x7f06100fc850 msgr2=0x7f0610110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 shutdown_connections 2024-04-14T15:27:27.232 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:27.231+0000 7f0615aa6640 1 -- 172.21.15.45:0/4257852180 wait complete. 2024-04-14T15:27:27.242 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 145360 2024-04-14T15:27:27.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[37795]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[37795]: osdmap e752: 8 total, 8 up, 8 in 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[37795]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[29624]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[29624]: osdmap e752: 8 total, 8 up, 8 in 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:27.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:27 smithi045 ceph-mon[29624]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:27.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:27 smithi071 ceph-mon[33150]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:27 smithi071 ceph-mon[33150]: osdmap e752: 8 total, 8 up, 8 in 2024-04-14T15:27:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:27 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/4257852180' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:27.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:27 smithi071 ceph-mon[33150]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]: dispatch 2024-04-14T15:27:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[37795]: pgmap v1187: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[37795]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:28.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[37795]: osdmap e753: 8 total, 8 up, 8 in 2024-04-14T15:27:28.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[29624]: pgmap v1187: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[29624]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:28.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:28 smithi045 ceph-mon[29624]: osdmap e753: 8 total, 8 up, 8 in 2024-04-14T15:27:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:28 smithi071 ceph-mon[33150]: pgmap v1187: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:28 smithi071 ceph-mon[33150]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "100"}]': finished 2024-04-14T15:27:28.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:28 smithi071 ceph-mon[33150]: osdmap e753: 8 total, 8 up, 8 in 2024-04-14T15:27:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:29.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:29 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:27:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:29.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:29 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:27:29.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:29.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:29 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:27:30.175 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 0 -ne 0 ']' 2024-04-14T15:27:30.175 INFO:tasks.workunit.client.0.smithi045.stderr:+ true 2024-04-14T15:27:30.176 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put twomore /etc/passwd 2024-04-14T15:27:30.211 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_bytes 100 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.298+0000 7fbda4836640 1 -- 172.21.15.45:0/3809739323 >> v1:172.21.15.45:6790/0 conn(0x7fbda0100c10 legacy=0x7fbda0100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.298+0000 7fbda4836640 1 -- 172.21.15.45:0/3809739323 shutdown_connections 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.298+0000 7fbda4836640 1 -- 172.21.15.45:0/3809739323 >> 172.21.15.45:0/3809739323 conn(0x7fbda00fc830 msgr2=0x7fbda00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.298+0000 7fbda4836640 1 -- 172.21.15.45:0/3809739323 shutdown_connections 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.298+0000 7fbda4836640 1 -- 172.21.15.45:0/3809739323 wait complete. 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbda4836640 1 Processor -- start 2024-04-14T15:27:30.301 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbda4836640 1 -- start start 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbda4836640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbda01a90c0 con 0x7fbda01017f0 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbda4836640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbda01aa300 con 0x7fbda0110ae0 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbda4836640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fbda01ab540 con 0x7fbda0100c10 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbd9e59c640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fbda0100c10 0x7fbda010f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:53846/0 (socket says 172.21.15.45:53846) 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbd9e59c640 1 -- 172.21.15.45:0/741452038 learned_addr learned my addr 172.21.15.45:0/741452038 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.299+0000 7fbd97fff640 1 --1- 172.21.15.45:0/741452038 >> v1:172.21.15.45:6789/0 conn(0x7fbda01017f0 0x7fbda01a3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51184/0 (socket says 172.21.15.45:51184) 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013433008 0 0) 0x7fbda01ab540 con 0x7fbda0100c10 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd8c003520 con 0x7fbda0100c10 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007061987 0 0) 0x7fbda01a90c0 con 0x7fbda01017f0 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda01ab540 con 0x7fbda01017f0 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1053659094 0 0) 0x7fbd8c003520 con 0x7fbda0100c10 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbda01a90c0 con 0x7fbda0100c10 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843756814 0 0) 0x7fbda01aa300 con 0x7fbda0110ae0 2024-04-14T15:27:30.302 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd8c003520 con 0x7fbda0110ae0 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fbd900045b0 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3947013806 0 0) 0x7fbda01ab540 con 0x7fbda01017f0 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbda01aa300 con 0x7fbda01017f0 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fbd88001060 con 0x7fbda01017f0 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 711670198 0 0) 0x7fbda01a90c0 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 >> v1:172.21.15.71:6789/0 conn(0x7fbda0110ae0 legacy=0x7fbda01a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 >> v1:172.21.15.45:6789/0 conn(0x7fbda01017f0 legacy=0x7fbda01a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda01ac780 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbda01a92f0 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbda01a97c0 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fbd90001b30 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.300+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fbd90004ca0 con 0x7fbda0100c10 2024-04-14T15:27:30.303 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.301+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd80005080 con 0x7fbda0100c10 2024-04-14T15:27:30.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.302+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fbd900037d0 con 0x7fbda0100c10 2024-04-14T15:27:30.304 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.302+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(754..754 src has 1..754) ==== 7786+0+0 (unknown 3068941309 0 0) 0x7fbd9008a920 con 0x7fbda0100c10 2024-04-14T15:27:30.306 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.307+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fbd90057510 con 0x7fbda0100c10 2024-04-14T15:27:30.492 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:30.492+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbd80005370 con 0x7fbda0100c10 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: pgmap v1189: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:27:30.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[29624]: osdmap e754: 8 total, 8 up, 8 in 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: pgmap v1189: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:27:30.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:30 smithi045 ceph-mon[37795]: osdmap e754: 8 total, 8 up, 8 in 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: pgmap v1189: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:27:30.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:30 smithi071 ceph-mon[33150]: osdmap e754: 8 total, 8 up, 8 in 2024-04-14T15:27:31.240 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:31.240+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v755) ==== 221+0+0 (unknown 2358833039 0 0) 0x7fbd90001100 con 0x7fbda0100c10 2024-04-14T15:27:31.356 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:31.357+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"} v 0) -- 0x7fbd80000c40 con 0x7fbda0100c10 2024-04-14T15:27:31.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:31 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:31.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:31 smithi045 ceph-mon[29624]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:31 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:31.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:31 smithi045 ceph-mon[37795]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:31 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:31.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:31 smithi071 ceph-mon[33150]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.241 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.241+0000 7fbd977fe640 1 -- 172.21.15.45:0/741452038 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v756) ==== 221+0+0 (unknown 52314427 0 0) 0x7fbd9005b1c0 con 0x7fbda0100c10 2024-04-14T15:27:32.241 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_bytes = 100 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:27:32.244 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.244+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 >> v1:172.21.15.45:6800/1718216366 conn(0x7fbd8c06fba0 legacy=0x7fbd8c072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:32.244 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.244+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 >> v1:172.21.15.45:6790/0 conn(0x7fbda0100c10 legacy=0x7fbda010f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:27:32.245 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.245+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 shutdown_connections 2024-04-14T15:27:32.245 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.245+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 >> 172.21.15.45:0/741452038 conn(0x7fbda00fc830 msgr2=0x7fbda0112f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:27:32.245 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.245+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 shutdown_connections 2024-04-14T15:27:32.245 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:27:32.245+0000 7fbda4836640 1 -- 172.21.15.45:0/741452038 wait complete. 2024-04-14T15:27:32.256 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 30 2024-04-14T15:27:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[29624]: pgmap v1191: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[29624]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:32.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[29624]: osdmap e755: 8 total, 8 up, 8 in 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[29624]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[37795]: pgmap v1191: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[37795]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[37795]: osdmap e755: 8 total, 8 up, 8 in 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.497 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:32 smithi045 ceph-mon[37795]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:32 smithi071 ceph-mon[33150]: pgmap v1191: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:32 smithi071 ceph-mon[33150]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:32 smithi071 ceph-mon[33150]: osdmap e755: 8 total, 8 up, 8 in 2024-04-14T15:27:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:32 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/741452038' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:32.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:32 smithi071 ceph-mon[33150]: from='client.57026 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]: dispatch 2024-04-14T15:27:33.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:33 smithi071 ceph-mon[33150]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:33.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:33 smithi071 ceph-mon[33150]: osdmap e756: 8 total, 8 up, 8 in 2024-04-14T15:27:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:33 smithi045 ceph-mon[29624]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:33.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:33 smithi045 ceph-mon[29624]: osdmap e756: 8 total, 8 up, 8 in 2024-04-14T15:27:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:33 smithi045 ceph-mon[37795]: from='client.57026 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "100"}]': finished 2024-04-14T15:27:33.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:33 smithi045 ceph-mon[37795]: osdmap e756: 8 total, 8 up, 8 in 2024-04-14T15:27:34.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:34 smithi071 ceph-mon[33150]: pgmap v1194: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-04-14T15:27:34.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:34.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:34 smithi045 ceph-mon[29624]: pgmap v1194: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-04-14T15:27:34.647 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:34 smithi045 ceph-mon[37795]: pgmap v1194: 176 pgs: 176 active+clean; 698 KiB data, 808 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-04-14T15:27:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:35 smithi071 ceph-mon[33150]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_bytes: 100 B) 2024-04-14T15:27:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:35 smithi071 ceph-mon[33150]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:35.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:35 smithi071 ceph-mon[33150]: osdmap e757: 8 total, 8 up, 8 in 2024-04-14T15:27:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[29624]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_bytes: 100 B) 2024-04-14T15:27:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[29624]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:35.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[29624]: osdmap e757: 8 total, 8 up, 8 in 2024-04-14T15:27:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[37795]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' is full (reached quota's max_bytes: 100 B) 2024-04-14T15:27:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[37795]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:27:35.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:35 smithi045 ceph-mon[37795]: osdmap e757: 8 total, 8 up, 8 in 2024-04-14T15:27:36.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:36 smithi071 ceph-mon[33150]: pgmap v1195: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:36.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:36 smithi045 ceph-mon[37795]: pgmap v1195: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:36 smithi045 ceph-mon[29624]: pgmap v1195: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:38.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:38 smithi071 ceph-mon[33150]: pgmap v1197: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:38.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:38 smithi045 ceph-mon[37795]: pgmap v1197: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:38.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:38 smithi045 ceph-mon[29624]: pgmap v1197: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-04-14T15:27:40.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:40 smithi071 ceph-mon[33150]: pgmap v1198: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-04-14T15:27:40.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:40 smithi045 ceph-mon[37795]: pgmap v1198: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-04-14T15:27:40.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:40 smithi045 ceph-mon[29624]: pgmap v1198: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-04-14T15:27:42.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:42 smithi071 ceph-mon[33150]: pgmap v1199: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-04-14T15:27:42.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:42 smithi045 ceph-mon[37795]: pgmap v1199: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-04-14T15:27:42.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:42 smithi045 ceph-mon[29624]: pgmap v1199: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-04-14T15:27:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:44 smithi071 ceph-mon[33150]: pgmap v1200: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-04-14T15:27:44.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:44 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:44.608 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:44 smithi045 ceph-mon[29624]: pgmap v1200: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-04-14T15:27:44.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:44 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:44 smithi045 ceph-mon[37795]: pgmap v1200: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-04-14T15:27:44.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:44 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:45.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:45.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:45.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:46.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:46 smithi071 ceph-mon[33150]: pgmap v1201: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:46.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:46 smithi045 ceph-mon[29624]: pgmap v1201: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:46.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:46 smithi045 ceph-mon[37795]: pgmap v1201: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:27:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:48.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:48 smithi071 ceph-mon[33150]: pgmap v1202: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-04-14T15:27:48.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:48 smithi045 ceph-mon[29624]: pgmap v1202: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-04-14T15:27:48.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:48 smithi045 ceph-mon[37795]: pgmap v1202: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 890 B/s rd, 0 op/s 2024-04-14T15:27:50.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:50 smithi071 ceph-mon[33150]: pgmap v1203: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:50.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:50 smithi045 ceph-mon[29624]: pgmap v1203: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:50.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:50 smithi045 ceph-mon[37795]: pgmap v1203: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:52.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:52 smithi071 ceph-mon[33150]: pgmap v1204: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:52.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:52 smithi045 ceph-mon[37795]: pgmap v1204: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:52.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:52 smithi045 ceph-mon[29624]: pgmap v1204: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:54.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:54 smithi071 ceph-mon[33150]: pgmap v1205: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:54.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:27:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:27:54.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:54 smithi045 ceph-mon[37795]: pgmap v1205: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:54.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:54 smithi045 ceph-mon[29624]: pgmap v1205: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:27:55.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:55 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:55.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:55 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:55.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:55 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:27:56.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:56 smithi071 ceph-mon[33150]: pgmap v1206: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:56.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:56 smithi045 ceph-mon[37795]: pgmap v1206: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:56 smithi045 ceph-mon[29624]: pgmap v1206: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:27:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:27:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:27:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:27:58.609 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:58 smithi071 ceph-mon[33150]: pgmap v1207: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:58.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:58 smithi045 ceph-mon[29624]: pgmap v1207: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:58.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:58 smithi045 ceph-mon[37795]: pgmap v1207: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:27:59.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:27:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:59.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:27:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:27:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:27:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:00.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:00 smithi071 ceph-mon[33150]: pgmap v1208: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:00.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:00 smithi045 ceph-mon[29624]: pgmap v1208: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:00.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:00 smithi045 ceph-mon[37795]: pgmap v1208: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:02.258 INFO:tasks.workunit.client.0.smithi045.stderr:+ pid=145576 2024-04-14T15:28:02.258 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_bytes 0 2024-04-14T15:28:02.259 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put two /etc/passwd 2024-04-14T15:28:02.359 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.354+0000 7fd81900f640 1 -- 172.21.15.45:0/2850935513 >> v1:172.21.15.45:6789/0 conn(0x7fd814103000 legacy=0x7fd8141033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:02.359 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.354+0000 7fd81900f640 1 -- 172.21.15.45:0/2850935513 shutdown_connections 2024-04-14T15:28:02.359 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.354+0000 7fd81900f640 1 -- 172.21.15.45:0/2850935513 >> 172.21.15.45:0/2850935513 conn(0x7fd814077a20 msgr2=0x7fd8140fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:02.360 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.354+0000 7fd81900f640 1 -- 172.21.15.45:0/2850935513 shutdown_connections 2024-04-14T15:28:02.360 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.354+0000 7fd81900f640 1 -- 172.21.15.45:0/2850935513 wait complete. 2024-04-14T15:28:02.360 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81900f640 1 Processor -- start 2024-04-14T15:28:02.360 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81900f640 1 -- start start 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81900f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8141a6ea0 con 0x7fd814103000 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81900f640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8141a80e0 con 0x7fd81410ae00 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81900f640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fd8141a9320 con 0x7fd81410e910 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd812d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd814103000 0x7fd81410a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57194/0 (socket says 172.21.15.45:57194) 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd81359e640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fd81410e910 0x7fd8141a55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60260/0 (socket says 172.21.15.45:60260) 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.355+0000 7fd812d9d640 1 -- 172.21.15.45:0/647625496 learned_addr learned my addr 172.21.15.45:0/647625496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158525244 0 0) 0x7fd8141a9320 con 0x7fd81410e910 2024-04-14T15:28:02.361 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd800003160 con 0x7fd81410e910 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233833245 0 0) 0x7fd8141a6ea0 con 0x7fd814103000 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8141a9320 con 0x7fd814103000 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079578940 0 0) 0x7fd8141a80e0 con 0x7fd81410ae00 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8141a6ea0 con 0x7fd81410ae00 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 954440875 0 0) 0x7fd800003160 con 0x7fd81410e910 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8141a80e0 con 0x7fd81410e910 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd804001db0 con 0x7fd81410e910 2024-04-14T15:28:02.362 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1215029539 0 0) 0x7fd8141a9320 con 0x7fd814103000 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd800003160 con 0x7fd814103000 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd80c004370 con 0x7fd814103000 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 14618348 0 0) 0x7fd8141a6ea0 con 0x7fd81410ae00 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd8141a9320 con 0x7fd81410ae00 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd7fc001830 con 0x7fd81410ae00 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2594500812 0 0) 0x7fd8141a80e0 con 0x7fd81410e910 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 >> v1:172.21.15.71:6789/0 conn(0x7fd81410ae00 legacy=0x7fd8141a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 >> v1:172.21.15.45:6789/0 conn(0x7fd814103000 legacy=0x7fd81410a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:02.363 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8141aa560 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd8141a70d0 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd8141a75a0 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fd804001f50 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.356+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fd804005dd0 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.357+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f4005080 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.358+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fd804003c20 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.358+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(757..757 src has 1..757) ==== 7786+0+0 (unknown 904676842 0 0) 0x7fd80408a930 con 0x7fd81410e910 2024-04-14T15:28:02.364 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.358+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=758}) -- 0x7fd8141a80e0 con 0x7fd81410e910 2024-04-14T15:28:02.365 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.363+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fd8040574c0 con 0x7fd81410e910 2024-04-14T15:28:02.550 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:02.550+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"} v 0) -- 0x7fd7f4005370 con 0x7fd81410e910 2024-04-14T15:28:02.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:02 smithi071 ceph-mon[33150]: pgmap v1209: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:02.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:02 smithi045 ceph-mon[29624]: pgmap v1209: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:02.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:02 smithi045 ceph-mon[37795]: pgmap v1209: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:03.312 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:03.311+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 10 ==== osd_map(758..758 src has 1..758) ==== 628+0+0 (unknown 1004985095 0 0) 0x7fd8040886f0 con 0x7fd81410e910 2024-04-14T15:28:03.312 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:03.311+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=759}) -- 0x7fd800003160 con 0x7fd81410e910 2024-04-14T15:28:03.315 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:03.315+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v758) ==== 217+0+0 (unknown 3187386461 0 0) 0x7fd80405b170 con 0x7fd81410e910 2024-04-14T15:28:03.435 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:03.435+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"} v 0) -- 0x7fd7f4000c40 con 0x7fd81410e910 2024-04-14T15:28:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:03 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:03.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:03 smithi071 ceph-mon[33150]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:03.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:03 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:03.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:03 smithi045 ceph-mon[29624]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:03 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:03.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:03 smithi045 ceph-mon[37795]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.312 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.312+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 12 ==== osd_map(759..759 src has 1..759) ==== 628+0+0 (unknown 4079188128 0 0) 0x7fd804052e00 con 0x7fd81410e910 2024-04-14T15:28:04.312 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.312+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=760}) -- 0x7fd8141a6ea0 con 0x7fd81410e910 2024-04-14T15:28:04.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.317+0000 7fd80bfff640 1 -- 172.21.15.45:0/647625496 <== mon.2 v1:172.21.15.45:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v759) ==== 217+0+0 (unknown 1822467475 0 0) 0x7fd80405fa30 con 0x7fd81410e910 2024-04-14T15:28:04.317 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 >> v1:172.21.15.45:6800/1718216366 conn(0x7fd80006f950 legacy=0x7fd800071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 >> v1:172.21.15.45:6790/0 conn(0x7fd81410e910 legacy=0x7fd8141a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 shutdown_connections 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 >> 172.21.15.45:0/647625496 conn(0x7fd814077a20 msgr2=0x7fd8141033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 shutdown_connections 2024-04-14T15:28:04.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.322+0000 7fd81900f640 1 -- 172.21.15.45:0/647625496 wait complete. 2024-04-14T15:28:04.332 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_objects 0 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- 172.21.15.45:0/4221670646 >> v1:172.21.15.45:6790/0 conn(0x7f6c2c100fb0 legacy=0x7f6c2c101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- 172.21.15.45:0/4221670646 shutdown_connections 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- 172.21.15.45:0/4221670646 >> 172.21.15.45:0/4221670646 conn(0x7f6c2c0fc850 msgr2=0x7f6c2c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- 172.21.15.45:0/4221670646 shutdown_connections 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- 172.21.15.45:0/4221670646 wait complete. 2024-04-14T15:28:04.421 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 Processor -- start 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.419+0000 7f6c3462d640 1 -- start start 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c3462d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6c2c1a6d80 con 0x7f6c2c109c30 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c3462d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6c2c1a7f80 con 0x7f6c2c10d7a0 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c3462d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6c2c1a91c0 con 0x7f6c2c100fb0 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c323c9640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6c2c100fb0 0x7f6c2c106190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60272/0 (socket says 172.21.15.45:60272) 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c323c9640 1 -- 172.21.15.45:0/2921620413 learned_addr learned my addr 172.21.15.45:0/2921620413 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188631255 0 0) 0x7f6c2c1a91c0 con 0x7f6c2c100fb0 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c18003520 con 0x7f6c2c100fb0 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1663622774 0 0) 0x7f6c2c1a6d80 con 0x7f6c2c109c30 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c2c1a91c0 con 0x7f6c2c109c30 2024-04-14T15:28:04.422 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096366653 0 0) 0x7f6c2c1a7f80 con 0x7f6c2c10d7a0 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c2c1a6d80 con 0x7f6c2c10d7a0 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3542042189 0 0) 0x7f6c2c1a91c0 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c2c1a7f80 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1752983001 0 0) 0x7f6c18003520 con 0x7f6c2c100fb0 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6c2c1a91c0 con 0x7f6c2c100fb0 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6c1c001870 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6c28004370 con 0x7f6c2c100fb0 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2469384658 0 0) 0x7f6c2c1a7f80 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 >> v1:172.21.15.45:6790/0 conn(0x7f6c2c100fb0 legacy=0x7f6c2c106190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 >> v1:172.21.15.71:6789/0 conn(0x7f6c2c10d7a0 legacy=0x7f6c2c1a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c1aa400 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c1a6fb0 con 0x7f6c2c109c30 2024-04-14T15:28:04.423 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1a7480 con 0x7f6c2c109c30 2024-04-14T15:28:04.424 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6c1c001c60 con 0x7f6c2c109c30 2024-04-14T15:28:04.424 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.420+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6c1c004a60 con 0x7f6c2c109c30 2024-04-14T15:28:04.424 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.421+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c10005080 con 0x7f6c2c109c30 2024-04-14T15:28:04.425 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.422+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f6c1c0038c0 con 0x7f6c2c109c30 2024-04-14T15:28:04.425 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.422+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(759..759 src has 1..759) ==== 7786+0+0 (unknown 897080259 0 0) 0x7f6c1c089490 con 0x7f6c2c109c30 2024-04-14T15:28:04.425 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.422+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=760}) -- 0x7f6c2c1a7f80 con 0x7f6c2c109c30 2024-04-14T15:28:04.427 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.427+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f6c1c056160 con 0x7f6c2c109c30 2024-04-14T15:28:04.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-mon[33150]: pgmap v1210: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-mon[33150]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-mon[33150]: osdmap e758: 8 total, 8 up, 8 in 2024-04-14T15:28:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:04 smithi071 ceph-mon[33150]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.615 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:04.616+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"} v 0) -- 0x7f6c10005370 con 0x7f6c2c109c30 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[37795]: pgmap v1210: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[37795]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[37795]: osdmap e758: 8 total, 8 up, 8 in 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[37795]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[29624]: pgmap v1210: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[29624]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[29624]: osdmap e758: 8 total, 8 up, 8 in 2024-04-14T15:28:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/647625496' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:04.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:04 smithi045 ceph-mon[29624]: from='client.57041 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:28:05.172 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:05.172+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v760) ==== 221+0+0 (unknown 2072526719 0 0) 0x7f6c1c059e10 con 0x7f6c2c109c30 2024-04-14T15:28:05.174 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:05.174+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 11 ==== osd_map(760..760 src has 1..760) ==== 628+0+0 (unknown 3467686689 0 0) 0x7f6c1c004a40 con 0x7f6c2c109c30 2024-04-14T15:28:05.174 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:05.174+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=761}) -- 0x7f6c2c1a6d80 con 0x7f6c2c109c30 2024-04-14T15:28:05.294 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:05.294+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"} v 0) -- 0x7f6c10000c40 con 0x7f6c2c109c30 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: osdmap e759: 8 total, 8 up, 8 in 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: osdmap e760: 8 total, 8 up, 8 in 2024-04-14T15:28:05.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:05 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: osdmap e759: 8 total, 8 up, 8 in 2024-04-14T15:28:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:05.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: osdmap e760: 8 total, 8 up, 8 in 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: from='client.57041 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: osdmap e759: 8 total, 8 up, 8 in 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' no longer out of quota; removing NO_QUOTA flag 2024-04-14T15:28:05.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:28:05.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:05.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: osdmap e760: 8 total, 8 up, 8 in 2024-04-14T15:28:05.748 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:05 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:28:06.175 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.174+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v761) ==== 221+0+0 (unknown 4225298078 0 0) 0x7f6c1c05e6d0 con 0x7f6c2c109c30 2024-04-14T15:28:06.175 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:28:06.175 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.175+0000 7f6c237fe640 1 -- 172.21.15.45:0/2921620413 <== mon.0 v1:172.21.15.45:6789/0 13 ==== osd_map(761..761 src has 1..761) ==== 628+0+0 (unknown 1666455677 0 0) 0x7f6c1c051aa0 con 0x7f6c2c109c30 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.178+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 >> v1:172.21.15.45:6800/1718216366 conn(0x7f6c1806f660 legacy=0x7f6c18071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.178+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 >> v1:172.21.15.45:6789/0 conn(0x7f6c2c109c30 legacy=0x7f6c2c1a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.179+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 shutdown_connections 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.179+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 >> 172.21.15.45:0/2921620413 conn(0x7f6c2c0fc850 msgr2=0x7f6c2c110cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.179+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 shutdown_connections 2024-04-14T15:28:06.179 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.179+0000 7f6c3462d640 1 -- 172.21.15.45:0/2921620413 wait complete. 2024-04-14T15:28:06.193 INFO:tasks.workunit.client.0.smithi045.stderr:+ wait 145576 2024-04-14T15:28:06.193 INFO:tasks.workunit.client.0.smithi045.stderr:+ '[' 0 -ne 0 ']' 2024-04-14T15:28:06.193 INFO:tasks.workunit.client.0.smithi045.stderr:+ true 2024-04-14T15:28:06.193 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put three /etc/passwd 2024-04-14T15:28:06.229 INFO:tasks.workunit.client.0.smithi045.stderr:++ uuidgen 2024-04-14T15:28:06.229 INFO:tasks.workunit.client.0.smithi045.stderr:+ pp=aac2e7fb-7118-4811-98cd-84307849713f 2024-04-14T15:28:06.229 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool create aac2e7fb-7118-4811-98cd-84307849713f 12 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.314+0000 7f359e3a5640 1 -- 172.21.15.45:0/3413258911 >> v1:172.21.15.45:6789/0 conn(0x7f359810ae10 legacy=0x7f359810d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.314+0000 7f359e3a5640 1 -- 172.21.15.45:0/3413258911 shutdown_connections 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.314+0000 7f359e3a5640 1 -- 172.21.15.45:0/3413258911 >> 172.21.15.45:0/3413258911 conn(0x7f35980fc850 msgr2=0x7f35980fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.314+0000 7f359e3a5640 1 -- 172.21.15.45:0/3413258911 shutdown_connections 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.314+0000 7f359e3a5640 1 -- 172.21.15.45:0/3413258911 wait complete. 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f359e3a5640 1 Processor -- start 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f359e3a5640 1 -- start start 2024-04-14T15:28:06.317 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f359e3a5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f359819ed30 con 0x7f3598103010 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f359e3a5640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f35981ae940 con 0x7f359810e920 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f359e3a5640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f35981afb20 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f3597fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3598103010 0x7f359810a690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57244/0 (socket says 172.21.15.45:57244) 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.315+0000 7f3597fff640 1 -- 172.21.15.45:0/3966710282 learned_addr learned my addr 172.21.15.45:0/3966710282 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425308574 0 0) 0x7f35981afb20 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3580003520 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431468898 0 0) 0x7f359819ed30 con 0x7f3598103010 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35981afb20 con 0x7f3598103010 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1347913122 0 0) 0x7f3580003520 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f359819ed30 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f358c004410 con 0x7f359810ae10 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2273440049 0 0) 0x7f35981afb20 con 0x7f3598103010 2024-04-14T15:28:06.318 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3580003520 con 0x7f3598103010 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f35880018b0 con 0x7f3598103010 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086488530 0 0) 0x7f35981ae940 con 0x7f359810e920 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35981afb20 con 0x7f359810e920 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3294151025 0 0) 0x7f359819ed30 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 >> v1:172.21.15.71:6789/0 conn(0x7f359810e920 legacy=0x7f359819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 >> v1:172.21.15.45:6789/0 conn(0x7f3598103010 legacy=0x7f359810a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35981b0d00 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f35981ad990 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f35981ade60 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f358c001b70 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.316+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f358c004c90 con 0x7f359810ae10 2024-04-14T15:28:06.319 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.317+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3584005080 con 0x7f359810ae10 2024-04-14T15:28:06.320 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.318+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f358c0037d0 con 0x7f359810ae10 2024-04-14T15:28:06.320 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.318+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(761..761 src has 1..761) ==== 7786+0+0 (unknown 915669276 0 0) 0x7f358c089720 con 0x7f359810ae10 2024-04-14T15:28:06.322 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.323+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f358c0563f0 con 0x7f359810ae10 2024-04-14T15:28:06.511 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:06.511+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12} v 0) -- 0x7f3584005370 con 0x7f359810ae10 2024-04-14T15:28:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:06 smithi071 ceph-mon[33150]: pgmap v1213: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:06 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:06.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:06 smithi071 ceph-mon[33150]: osdmap e761: 8 total, 8 up, 8 in 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[37795]: pgmap v1213: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[37795]: osdmap e761: 8 total, 8 up, 8 in 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[29624]: pgmap v1213: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2921620413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:28:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-mon[29624]: osdmap e761: 8 total, 8 up, 8 in 2024-04-14T15:28:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:07.339 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.339+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]=0 pool 'aac2e7fb-7118-4811-98cd-84307849713f' created v762) ==== 176+0+0 (unknown 874175863 0 0) 0x7f358c05a0a0 con 0x7f359810ae10 2024-04-14T15:28:07.459 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.459+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12} v 0) -- 0x7f3584001170 con 0x7f359810ae10 2024-04-14T15:28:07.460 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.460+0000 7f35957fa640 1 -- 172.21.15.45:0/3966710282 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]=0 pool 'aac2e7fb-7118-4811-98cd-84307849713f' already exists v762) ==== 183+0+0 (unknown 1854814335 0 0) 0x7f3584001170 con 0x7f359810ae10 2024-04-14T15:28:07.460 INFO:tasks.workunit.client.0.smithi045.stderr:pool 'aac2e7fb-7118-4811-98cd-84307849713f' already exists 2024-04-14T15:28:07.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.462+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 >> v1:172.21.15.45:6800/1718216366 conn(0x7f358006f9c0 legacy=0x7f3580071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:07.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.462+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 >> v1:172.21.15.45:6790/0 conn(0x7f359810ae10 legacy=0x7f359819dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:07.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.462+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 shutdown_connections 2024-04-14T15:28:07.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.462+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 >> 172.21.15.45:0/3966710282 conn(0x7f35980fc850 msgr2=0x7f3598110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:07.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.463+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 shutdown_connections 2024-04-14T15:28:07.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.463+0000 7f359e3a5640 1 -- 172.21.15.45:0/3966710282 wait complete. 2024-04-14T15:28:07.471 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool application enable aac2e7fb-7118-4811-98cd-84307849713f rados 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 -- 172.21.15.45:0/858159723 >> v1:172.21.15.45:6789/0 conn(0x7fa8a8100c00 legacy=0x7fa8a8100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 -- 172.21.15.45:0/858159723 shutdown_connections 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 -- 172.21.15.45:0/858159723 >> 172.21.15.45:0/858159723 conn(0x7fa8a8077a20 msgr2=0x7fa8a80fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 -- 172.21.15.45:0/858159723 shutdown_connections 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 -- 172.21.15.45:0/858159723 wait complete. 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.555+0000 7fa8ade2d640 1 Processor -- start 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8ade2d640 1 -- start start 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8ade2d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa8a81a8ea0 con 0x7fa8a8110ad0 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8ade2d640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa8a81aa0e0 con 0x7fa8a81017e0 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8ade2d640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fa8a81ab320 con 0x7fa8a8100c00 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a77fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fa8a8100c00 0x7fa8a810efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60350/0 (socket says 172.21.15.45:60350) 2024-04-14T15:28:07.556 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a77fe640 1 -- 172.21.15.45:0/2655600064 learned_addr learned my addr 172.21.15.45:0/2655600064 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655955393 0 0) 0x7fa8a81a8ea0 con 0x7fa8a8110ad0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa890003520 con 0x7fa8a8110ad0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282688670 0 0) 0x7fa8a81ab320 con 0x7fa8a8100c00 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a81a8ea0 con 0x7fa8a8100c00 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181185203 0 0) 0x7fa8a81aa0e0 con 0x7fa8a81017e0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a81ab320 con 0x7fa8a81017e0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1324497323 0 0) 0x7fa8a81a8ea0 con 0x7fa8a8100c00 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.556+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa8a81aa0e0 con 0x7fa8a8100c00 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2675446291 0 0) 0x7fa890003520 con 0x7fa8a8110ad0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa8a81a8ea0 con 0x7fa8a8110ad0 2024-04-14T15:28:07.559 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa898004370 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa8a0001db0 con 0x7fa8a8110ad0 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 124045923 0 0) 0x7fa8a81ab320 con 0x7fa8a81017e0 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa890003520 con 0x7fa8a81017e0 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa89c001830 con 0x7fa8a81017e0 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1001635986 0 0) 0x7fa8a81aa0e0 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 >> v1:172.21.15.71:6789/0 conn(0x7fa8a81017e0 legacy=0x7fa8a81a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 >> v1:172.21.15.45:6789/0 conn(0x7fa8a8110ad0 legacy=0x7fa8a81a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a81ac560 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a81ab550 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa8a81aba20 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fa898001b50 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.557+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fa898004d70 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.558+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa894005080 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.558+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fa898004d50 con 0x7fa8a8100c00 2024-04-14T15:28:07.560 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.559+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(762..762 src has 1..762) ==== 8161+0+0 (unknown 4082028797 0 0) 0x7fa89808a890 con 0x7fa8a8100c00 2024-04-14T15:28:07.563 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.563+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fa8980572e0 con 0x7fa8a8100c00 2024-04-14T15:28:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:07 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:07 smithi071 ceph-mon[33150]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:07 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:07 smithi045 ceph-mon[37795]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:07 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:07 smithi045 ceph-mon[29624]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:07.750 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:07.750+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"} v 0) -- 0x7fa894005370 con 0x7fa8a8100c00 2024-04-14T15:28:08.340 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:08.340+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]=0 enabled application 'rados' on pool 'aac2e7fb-7118-4811-98cd-84307849713f' v763) ==== 213+0+0 (unknown 2425050921 0 0) 0x7fa89805af90 con 0x7fa8a8100c00 2024-04-14T15:28:08.461 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:08.461+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"} v 0) -- 0x7fa894001090 con 0x7fa8a8100c00 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: pgmap v1216: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]': finished 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: osdmap e762: 8 total, 8 up, 8 in 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:08.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:08 smithi071 ceph-mon[33150]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: pgmap v1216: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]': finished 2024-04-14T15:28:08.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: osdmap e762: 8 total, 8 up, 8 in 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[29624]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: pgmap v1216: 176 pgs: 176 active+clean; 721 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: from='client.57062 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]': finished 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: osdmap e762: 8 total, 8 up, 8 in 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/3966710282' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: from='client.57062 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pg_num": 12}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:08.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:08 smithi045 ceph-mon[37795]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.354 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.354+0000 7fa8a4ff9640 1 -- 172.21.15.45:0/2655600064 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]=0 enabled application 'rados' on pool 'aac2e7fb-7118-4811-98cd-84307849713f' v764) ==== 213+0+0 (unknown 913217435 0 0) 0x7fa89805f850 con 0x7fa8a8100c00 2024-04-14T15:28:09.354 INFO:tasks.workunit.client.0.smithi045.stderr:enabled application 'rados' on pool 'aac2e7fb-7118-4811-98cd-84307849713f' 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 >> v1:172.21.15.45:6800/1718216366 conn(0x7fa89006f8e0 legacy=0x7fa890071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 >> v1:172.21.15.45:6790/0 conn(0x7fa8a8100c00 legacy=0x7fa8a810efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 shutdown_connections 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 >> 172.21.15.45:0/2655600064 conn(0x7fa8a8077a20 msgr2=0x7fa8a8112f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 shutdown_connections 2024-04-14T15:28:09.358 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.357+0000 7fa8ade2d640 1 -- 172.21.15.45:0/2655600064 wait complete. 2024-04-14T15:28:09.369 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota aac2e7fb-7118-4811-98cd-84307849713f max_objects 10 2024-04-14T15:28:09.461 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.458+0000 7f70cbf1e640 1 -- 172.21.15.45:0/1322119972 >> v1:172.21.15.45:6790/0 conn(0x7f70c4100fb0 legacy=0x7f70c4101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:09.461 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- 172.21.15.45:0/1322119972 shutdown_connections 2024-04-14T15:28:09.461 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- 172.21.15.45:0/1322119972 >> 172.21.15.45:0/1322119972 conn(0x7f70c40fc850 msgr2=0x7f70c40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:09.461 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- 172.21.15.45:0/1322119972 shutdown_connections 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- 172.21.15.45:0/1322119972 wait complete. 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 Processor -- start 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- start start 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f70c41a6d60 con 0x7f70c4109c30 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f70c41a7fa0 con 0x7f70c4100fb0 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70cbf1e640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f70c41a91e0 con 0x7f70c410d7a0 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.459+0000 7f70ca4bb640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f70c410d7a0 0x7f70c41a5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:60374/0 (socket says 172.21.15.45:60374) 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70ca4bb640 1 -- 172.21.15.45:0/192132099 learned_addr learned my addr 172.21.15.45:0/192132099 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6252318 0 0) 0x7f70c41a91e0 con 0x7f70c410d7a0 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70b0003520 con 0x7f70c410d7a0 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627130555 0 0) 0x7f70c41a6d60 con 0x7f70c4109c30 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c41a91e0 con 0x7f70c4109c30 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682309772 0 0) 0x7f70c41a7fa0 con 0x7f70c4100fb0 2024-04-14T15:28:09.462 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c41a6d60 con 0x7f70c4100fb0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1995181961 0 0) 0x7f70b0003520 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f70c41a7fa0 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f70bc001d30 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1250661815 0 0) 0x7f70c41a91e0 con 0x7f70c4109c30 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f70b0003520 con 0x7f70c4109c30 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 476430168 0 0) 0x7f70c41a6d60 con 0x7f70c4100fb0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f70c41a91e0 con 0x7f70c4100fb0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f70b40017b0 con 0x7f70c4109c30 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f70c0004370 con 0x7f70c4100fb0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4012365649 0 0) 0x7f70c41a7fa0 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 >> v1:172.21.15.71:6789/0 conn(0x7f70c4100fb0 legacy=0x7f70c41061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 >> v1:172.21.15.45:6789/0 conn(0x7f70c4109c30 legacy=0x7f70c41a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41aa420 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41a6f90 con 0x7f70c410d7a0 2024-04-14T15:28:09.463 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f70c41a7460 con 0x7f70c410d7a0 2024-04-14T15:28:09.464 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f70bc003e00 con 0x7f70c410d7a0 2024-04-14T15:28:09.464 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.460+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f70bc005f50 con 0x7f70c410d7a0 2024-04-14T15:28:09.464 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.461+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a8005080 con 0x7f70c410d7a0 2024-04-14T15:28:09.464 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.462+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f70bc003970 con 0x7f70c410d7a0 2024-04-14T15:28:09.464 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.462+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(764..764 src has 1..764) ==== 8174+0+0 (unknown 923487468 0 0) 0x7f70bc08ac40 con 0x7f70c410d7a0 2024-04-14T15:28:09.466 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.467+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f70bc057690 con 0x7f70c410d7a0 2024-04-14T15:28:09.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:09 smithi071 ceph-mon[33150]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:09 smithi071 ceph-mon[33150]: osdmap e763: 8 total, 8 up, 8 in 2024-04-14T15:28:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:09 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:09 smithi071 ceph-mon[33150]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.658 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:09.658+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"} v 0) -- 0x7f70a8005370 con 0x7f70c410d7a0 2024-04-14T15:28:09.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[29624]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:09.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[29624]: osdmap e763: 8 total, 8 up, 8 in 2024-04-14T15:28:09.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[29624]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[37795]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[37795]: osdmap e763: 8 total, 8 up, 8 in 2024-04-14T15:28:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2655600064' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:09.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:09 smithi045 ceph-mon[37795]: from='client.57068 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]: dispatch 2024-04-14T15:28:10.354 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:10.354+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool aac2e7fb-7118-4811-98cd-84307849713f v765) ==== 223+0+0 (unknown 1076897144 0 0) 0x7f70bc05b340 con 0x7f70c410d7a0 2024-04-14T15:28:10.474 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:10.474+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"} v 0) -- 0x7f70a8001090 con 0x7f70c410d7a0 2024-04-14T15:28:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:10 smithi071 ceph-mon[33150]: pgmap v1219: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:10 smithi071 ceph-mon[33150]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:10 smithi071 ceph-mon[33150]: osdmap e764: 8 total, 8 up, 8 in 2024-04-14T15:28:10.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:10 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:10.608 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:10 smithi071 ceph-mon[33150]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[29624]: pgmap v1219: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:10.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[29624]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[29624]: osdmap e764: 8 total, 8 up, 8 in 2024-04-14T15:28:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:10.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[29624]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[37795]: pgmap v1219: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:10.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[37795]: from='client.57068 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "app": "rados"}]': finished 2024-04-14T15:28:10.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[37795]: osdmap e764: 8 total, 8 up, 8 in 2024-04-14T15:28:10.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:10.748 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:10 smithi045 ceph-mon[37795]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.366 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.366+0000 7f70baffd640 1 -- 172.21.15.45:0/192132099 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool aac2e7fb-7118-4811-98cd-84307849713f v766) ==== 223+0+0 (unknown 2617166568 0 0) 0x7f70bc05fc00 con 0x7f70c410d7a0 2024-04-14T15:28:11.366 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_objects = 10 for pool aac2e7fb-7118-4811-98cd-84307849713f 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 >> v1:172.21.15.45:6800/1718216366 conn(0x7f70b006f910 legacy=0x7f70b0071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 >> v1:172.21.15.45:6790/0 conn(0x7f70c410d7a0 legacy=0x7f70c41a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 shutdown_connections 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 >> 172.21.15.45:0/192132099 conn(0x7f70c40fc850 msgr2=0x7f70c406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 shutdown_connections 2024-04-14T15:28:11.370 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:28:11.370+0000 7f70cbf1e640 1 -- 172.21.15.45:0/192132099 wait complete. 2024-04-14T15:28:11.381 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 30 2024-04-14T15:28:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[37795]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[37795]: osdmap e765: 8 total, 8 up, 8 in 2024-04-14T15:28:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[37795]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[29624]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[29624]: osdmap e765: 8 total, 8 up, 8 in 2024-04-14T15:28:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:11 smithi045 ceph-mon[29624]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:11 smithi071 ceph-mon[33150]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:11.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:11 smithi071 ceph-mon[33150]: osdmap e765: 8 total, 8 up, 8 in 2024-04-14T15:28:11.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:11 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/192132099' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:11.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:11 smithi071 ceph-mon[33150]: from='client.57074 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]: dispatch 2024-04-14T15:28:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[37795]: pgmap v1222: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[37795]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:12.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[37795]: osdmap e766: 8 total, 8 up, 8 in 2024-04-14T15:28:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[29624]: pgmap v1222: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[29624]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:12.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:12 smithi045 ceph-mon[29624]: osdmap e766: 8 total, 8 up, 8 in 2024-04-14T15:28:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:12 smithi071 ceph-mon[33150]: pgmap v1222: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-04-14T15:28:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:12 smithi071 ceph-mon[33150]: from='client.57074 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "field": "max_objects", "val": "10"}]': finished 2024-04-14T15:28:12.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:12 smithi071 ceph-mon[33150]: osdmap e766: 8 total, 8 up, 8 in 2024-04-14T15:28:14.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:14 smithi071 ceph-mon[33150]: pgmap v1224: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-04-14T15:28:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:14.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:14 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[29624]: pgmap v1224: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-04-14T15:28:14.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:14.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[37795]: pgmap v1224: 188 pgs: 188 active+clean; 744 KiB data, 809 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.5 KiB/s wr, 1 op/s 2024-04-14T15:28:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:14.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:14 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:15.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:15 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:15.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:15 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:15.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:15 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:16.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:16 smithi045 ceph-mon[29624]: pgmap v1225: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:16.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:16 smithi045 ceph-mon[37795]: pgmap v1225: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:16.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:16 smithi071 ceph-mon[33150]: pgmap v1225: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:18.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:18 smithi045 ceph-mon[29624]: pgmap v1226: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 698 B/s rd, 0 op/s 2024-04-14T15:28:18.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:18 smithi045 ceph-mon[37795]: pgmap v1226: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 698 B/s rd, 0 op/s 2024-04-14T15:28:18.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:18 smithi071 ceph-mon[33150]: pgmap v1226: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 698 B/s rd, 0 op/s 2024-04-14T15:28:20.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:20 smithi045 ceph-mon[29624]: pgmap v1227: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:20.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:20 smithi045 ceph-mon[37795]: pgmap v1227: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:20.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:20 smithi071 ceph-mon[33150]: pgmap v1227: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:22.659 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:22 smithi071 ceph-mon[33150]: pgmap v1228: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:22.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:22 smithi045 ceph-mon[29624]: pgmap v1228: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:22.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:22 smithi045 ceph-mon[37795]: pgmap v1228: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:24.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:24 smithi071 ceph-mon[33150]: pgmap v1229: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-04-14T15:28:24.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:24.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:24 smithi045 ceph-mon[37795]: pgmap v1229: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-04-14T15:28:24.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:24 smithi045 ceph-mon[29624]: pgmap v1229: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-04-14T15:28:25.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:25 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:25.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:25 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:25.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:25 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:26.728 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:26 smithi045 ceph-mon[37795]: pgmap v1230: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:26.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:26 smithi045 ceph-mon[29624]: pgmap v1230: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:26.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:26 smithi071 ceph-mon[33150]: pgmap v1230: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:27.746 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:27 smithi045 ceph-mon[37795]: pgmap v1231: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:28:27.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:27 smithi045 ceph-mon[29624]: pgmap v1231: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:28:27.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:27 smithi071 ceph-mon[33150]: pgmap v1231: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-04-14T15:28:28.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:28.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:28.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:29.856 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:29 smithi071 ceph-mon[33150]: pgmap v1232: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:29.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:29 smithi045 ceph-mon[37795]: pgmap v1232: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:29.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:29 smithi045 ceph-mon[29624]: pgmap v1232: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:28:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:28:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:28:30.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:28:30.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:28:30.999 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:31.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:31 smithi071 ceph-mon[33150]: pgmap v1233: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:31.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:31 smithi045 ceph-mon[37795]: pgmap v1233: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:31.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:31 smithi045 ceph-mon[29624]: pgmap v1233: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:33.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:33 smithi045 ceph-mon[37795]: pgmap v1234: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:33.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:33 smithi045 ceph-mon[29624]: pgmap v1234: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:33 smithi071 ceph-mon[33150]: pgmap v1234: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:34.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:35.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:34 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:35.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:34 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:35.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:34 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:35 smithi071 ceph-mon[33150]: pgmap v1235: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:36.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:35 smithi045 ceph-mon[37795]: pgmap v1235: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:36.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:35 smithi045 ceph-mon[29624]: pgmap v1235: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:38.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:37 smithi071 ceph-mon[33150]: pgmap v1236: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:37 smithi045 ceph-mon[37795]: pgmap v1236: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:38.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:37 smithi045 ceph-mon[29624]: pgmap v1236: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:40.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:39 smithi071 ceph-mon[33150]: pgmap v1237: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:39 smithi045 ceph-mon[37795]: pgmap v1237: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:40.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:39 smithi045 ceph-mon[29624]: pgmap v1237: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:41.383 INFO:tasks.workunit.client.0.smithi045.stderr:++ seq 1 10 2024-04-14T15:28:41.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.383 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj1 /etc/passwd 2024-04-14T15:28:41.423 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.423 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj2 /etc/passwd 2024-04-14T15:28:41.459 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.459 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj3 /etc/passwd 2024-04-14T15:28:41.497 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.497 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj4 /etc/passwd 2024-04-14T15:28:41.527 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.527 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj5 /etc/passwd 2024-04-14T15:28:41.561 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.561 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj6 /etc/passwd 2024-04-14T15:28:41.594 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.594 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj7 /etc/passwd 2024-04-14T15:28:41.626 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.626 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj8 /etc/passwd 2024-04-14T15:28:41.659 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.659 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj9 /etc/passwd 2024-04-14T15:28:41.691 INFO:tasks.workunit.client.0.smithi045.stderr:+ for f in `seq 1 10` 2024-04-14T15:28:41.692 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p aac2e7fb-7118-4811-98cd-84307849713f put obj10 /etc/passwd 2024-04-14T15:28:41.725 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 30 2024-04-14T15:28:42.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:41 smithi071 ceph-mon[33150]: pgmap v1238: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:41 smithi045 ceph-mon[37795]: pgmap v1238: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:42.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:41 smithi045 ceph-mon[29624]: pgmap v1238: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:43 smithi071 ceph-mon[33150]: pgmap v1239: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:44.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:43 smithi045 ceph-mon[37795]: pgmap v1239: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:44.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:43 smithi045 ceph-mon[29624]: pgmap v1239: 188 pgs: 188 active+clean; 744 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:28:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:44.766 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:45.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:44 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:45.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:44 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:45.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:44 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:45 smithi071 ceph-mon[33150]: pgmap v1240: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-04-14T15:28:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:45 smithi071 ceph-mon[33150]: pool 'aac2e7fb-7118-4811-98cd-84307849713f' is full (reached quota's max_objects: 10) 2024-04-14T15:28:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:45 smithi071 ceph-mon[33150]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:28:46.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:45 smithi071 ceph-mon[33150]: osdmap e767: 8 total, 8 up, 8 in 2024-04-14T15:28:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[37795]: pgmap v1240: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-04-14T15:28:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[37795]: pool 'aac2e7fb-7118-4811-98cd-84307849713f' is full (reached quota's max_objects: 10) 2024-04-14T15:28:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[37795]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:28:46.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[37795]: osdmap e767: 8 total, 8 up, 8 in 2024-04-14T15:28:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[29624]: pgmap v1240: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-04-14T15:28:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[29624]: pool 'aac2e7fb-7118-4811-98cd-84307849713f' is full (reached quota's max_objects: 10) 2024-04-14T15:28:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[29624]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-04-14T15:28:46.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:45 smithi045 ceph-mon[29624]: osdmap e767: 8 total, 8 up, 8 in 2024-04-14T15:28:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:48.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:47 smithi071 ceph-mon[33150]: pgmap v1242: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:48.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:47 smithi045 ceph-mon[37795]: pgmap v1242: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:47 smithi045 ceph-mon[29624]: pgmap v1242: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:50.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:49 smithi071 ceph-mon[33150]: pgmap v1243: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:50.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:49 smithi045 ceph-mon[37795]: pgmap v1243: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:50.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:49 smithi045 ceph-mon[29624]: pgmap v1243: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:51 smithi071 ceph-mon[33150]: pgmap v1244: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:52.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:51 smithi045 ceph-mon[37795]: pgmap v1244: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:52.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:51 smithi045 ceph-mon[29624]: pgmap v1244: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:54.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:53 smithi071 ceph-mon[33150]: pgmap v1245: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:54.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:53 smithi045 ceph-mon[37795]: pgmap v1245: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:54.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:53 smithi045 ceph-mon[29624]: pgmap v1245: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-04-14T15:28:54.784 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:28:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:28:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:54 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:55.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:54 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:55.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:54 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:28:56.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:55 smithi071 ceph-mon[33150]: pgmap v1246: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:56.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:55 smithi045 ceph-mon[29624]: pgmap v1246: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:56.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:55 smithi045 ceph-mon[37795]: pgmap v1246: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-04-14T15:28:56.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:28:56 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:28:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:28:58.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:57 smithi071 ceph-mon[33150]: pgmap v1247: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-04-14T15:28:58.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:57 smithi045 ceph-mon[29624]: pgmap v1247: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-04-14T15:28:58.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:57 smithi045 ceph-mon[37795]: pgmap v1247: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-04-14T15:28:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:59.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[29624]: pgmap v1248: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:59.747 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:28:59 smithi045 ceph-mon[37795]: pgmap v1248: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:28:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:28:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:59 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:28:59.857 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:28:59 smithi071 ceph-mon[33150]: pgmap v1248: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:02.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:01 smithi071 ceph-mon[33150]: pgmap v1249: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:02.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:01 smithi045 ceph-mon[29624]: pgmap v1249: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:02.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:01 smithi045 ceph-mon[37795]: pgmap v1249: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:04.048 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:03 smithi045 ceph-mon[37795]: pgmap v1250: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:04.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:03 smithi045 ceph-mon[29624]: pgmap v1250: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:04.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:03 smithi071 ceph-mon[33150]: pgmap v1250: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:04.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:29:04 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:29:05.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:04 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:05.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:04 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:05.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:04 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:06.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:05 smithi071 ceph-mon[33150]: pgmap v1251: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:06.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:05 smithi045 ceph-mon[29624]: pgmap v1251: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:06.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:05 smithi045 ceph-mon[37795]: pgmap v1251: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:06.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:06 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:29:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:29:08.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:07 smithi071 ceph-mon[33150]: pgmap v1252: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:08.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:07 smithi045 ceph-mon[37795]: pgmap v1252: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:08.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:07 smithi045 ceph-mon[29624]: pgmap v1252: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:10.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:09 smithi071 ceph-mon[33150]: pgmap v1253: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:10.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:09 smithi045 ceph-mon[29624]: pgmap v1253: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:10.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:09 smithi045 ceph-mon[37795]: pgmap v1253: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:11.726 INFO:tasks.workunit.client.0.smithi045.stderr:+ rados -p c485d52f-0910-43c8-a4a6-e912c4af97a1 put threemore /etc/passwd 2024-04-14T15:29:11.766 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_bytes 0 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.854+0000 7f9acca08640 1 -- 172.21.15.45:0/1409169015 >> v1:172.21.15.45:6789/0 conn(0x7f9ac810ae10 legacy=0x7f9ac810d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.854+0000 7f9acca08640 1 -- 172.21.15.45:0/1409169015 shutdown_connections 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.854+0000 7f9acca08640 1 -- 172.21.15.45:0/1409169015 >> 172.21.15.45:0/1409169015 conn(0x7f9ac80fc850 msgr2=0x7f9ac80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.855+0000 7f9acca08640 1 -- 172.21.15.45:0/1409169015 shutdown_connections 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.855+0000 7f9acca08640 1 -- 172.21.15.45:0/1409169015 wait complete. 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.855+0000 7f9acca08640 1 Processor -- start 2024-04-14T15:29:11.855 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.855+0000 7f9acca08640 1 -- start start 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9acca08640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9ac81a9110 con 0x7f9ac8103010 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9acca08640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9ac81aa310 con 0x7f9ac810e920 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9acca08640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f9ac81ab550 con 0x7f9ac810ae10 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9ac659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ac8103010 0x7f9ac810d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58922/0 (socket says 172.21.15.45:58922) 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9ac659c640 1 -- 172.21.15.45:0/1826922947 learned_addr learned my addr 172.21.15.45:0/1826922947 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abffff640 1 --1- 172.21.15.45:0/1826922947 >> v1:172.21.15.45:6790/0 conn(0x7f9ac810ae10 0x7f9ac810dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33898/0 (socket says 172.21.15.45:33898) 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719386250 0 0) 0x7f9ac81a9110 con 0x7f9ac8103010 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ab4003520 con 0x7f9ac8103010 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933421900 0 0) 0x7f9ac81ab550 con 0x7f9ac810ae10 2024-04-14T15:29:11.858 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac81a9110 con 0x7f9ac810ae10 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178072708 0 0) 0x7f9ac81aa310 con 0x7f9ac810e920 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ac81ab550 con 0x7f9ac810e920 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3116778141 0 0) 0x7f9ac81a9110 con 0x7f9ac810ae10 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ac81aa310 con 0x7f9ac810ae10 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ab00043b0 con 0x7f9ac810ae10 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 331363930 0 0) 0x7f9ab4003520 con 0x7f9ac8103010 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ac81a9110 con 0x7f9ac8103010 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ab80018b0 con 0x7f9ac8103010 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 275614895 0 0) 0x7f9ac81ab550 con 0x7f9ac810e920 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9ab4003520 con 0x7f9ac810e920 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ac00013f0 con 0x7f9ac810e920 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1877365011 0 0) 0x7f9ac81aa310 con 0x7f9ac810ae10 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 >> v1:172.21.15.71:6789/0 conn(0x7f9ac810e920 legacy=0x7f9ac81a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 >> v1:172.21.15.45:6789/0 conn(0x7f9ac8103010 legacy=0x7f9ac810d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:11.859 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac81ac790 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.856+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac81ab780 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.857+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9ac81abc50 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.857+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f9ab0001ab0 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.857+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f9ab0004d40 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.857+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8005080 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.858+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f9ab0004f80 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.858+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(767..767 src has 1..767) ==== 8174+0+0 (unknown 1647631203 0 0) 0x7f9ab008a9e0 con 0x7f9ac810ae10 2024-04-14T15:29:11.860 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.859+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=768}) -- 0x7f9ac81aa310 con 0x7f9ac810ae10 2024-04-14T15:29:11.863 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:11.863+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f9ab0057450 con 0x7f9ac810ae10 2024-04-14T15:29:12.048 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:12.048+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"} v 0) -- 0x7f9aa8005370 con 0x7f9ac810ae10 2024-04-14T15:29:12.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:11 smithi071 ceph-mon[33150]: pgmap v1254: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:12.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:11 smithi045 ceph-mon[29624]: pgmap v1254: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:12.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:11 smithi045 ceph-mon[37795]: pgmap v1254: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:12.777 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:12.777+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 10 ==== osd_map(768..768 src has 1..768) ==== 628+0+0 (unknown 1227293565 0 0) 0x7f9ab0088680 con 0x7f9ac810ae10 2024-04-14T15:29:12.777 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:12.777+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=769}) -- 0x7f9ac81a9110 con 0x7f9ac810ae10 2024-04-14T15:29:12.779 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:12.780+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v768) ==== 217+0+0 (unknown 1224734371 0 0) 0x7f9ab005b100 con 0x7f9ac810ae10 2024-04-14T15:29:12.895 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:12.895+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"} v 0) -- 0x7f9aa8000c40 con 0x7f9ac810ae10 2024-04-14T15:29:13.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:12 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:12 smithi071 ceph-mon[33150]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:12 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:12 smithi045 ceph-mon[29624]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:12 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:12 smithi045 ceph-mon[37795]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:13.778 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.778+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 12 ==== osd_map(769..769 src has 1..769) ==== 628+0+0 (unknown 1786124558 0 0) 0x7f9ab0053330 con 0x7f9ac810ae10 2024-04-14T15:29:13.778 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.778+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=770}) -- 0x7f9ac81ab550 con 0x7f9ac810ae10 2024-04-14T15:29:13.781 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.781+0000 7f9abf7fe640 1 -- 172.21.15.45:0/1826922947 <== mon.2 v1:172.21.15.45:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v769) ==== 217+0+0 (unknown 2577726317 0 0) 0x7f9ab005f9c0 con 0x7f9ac810ae10 2024-04-14T15:29:13.781 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_bytes = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.785+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 >> v1:172.21.15.45:6800/1718216366 conn(0x7f9ab406f8b0 legacy=0x7f9ab4071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.785+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 >> v1:172.21.15.45:6790/0 conn(0x7f9ac810ae10 legacy=0x7f9ac810dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.786+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 shutdown_connections 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.786+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 >> 172.21.15.45:0/1826922947 conn(0x7f9ac80fc850 msgr2=0x7f9ac80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.786+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 shutdown_connections 2024-04-14T15:29:13.786 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.786+0000 7f9acca08640 1 -- 172.21.15.45:0/1826922947 wait complete. 2024-04-14T15:29:13.797 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool set-quota c485d52f-0910-43c8-a4a6-e912c4af97a1 max_objects 0 2024-04-14T15:29:13.886 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1763964383 >> v1:172.21.15.45:6789/0 conn(0x7f1ab010cf20 legacy=0x7f1ab010f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:13.886 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1763964383 shutdown_connections 2024-04-14T15:29:13.886 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1763964383 >> 172.21.15.45:0/1763964383 conn(0x7f1ab0077150 msgr2=0x7f1ab00fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:13.886 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1763964383 shutdown_connections 2024-04-14T15:29:13.886 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1763964383 wait complete. 2024-04-14T15:29:13.887 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.886+0000 7f1ab5f1e640 1 Processor -- start 2024-04-14T15:29:13.887 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1ab5f1e640 1 -- start start 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1ab5f1e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1ab01a6f00 con 0x7f1ab0106580 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1ab5f1e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1ab01a8100 con 0x7f1ab010cf20 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1ab5f1e640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f1ab01a9340 con 0x7f1ab0109320 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aaf7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ab0106580 0x7f1ab0102100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58942/0 (socket says 172.21.15.45:58942) 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aaeffd640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f1ab0109320 0x7f1ab0102810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:33918/0 (socket says 172.21.15.45:33918) 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aaf7fe640 1 -- 172.21.15.45:0/1386457216 learned_addr learned my addr 172.21.15.45:0/1386457216 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 747353684 0 0) 0x7f1ab01a9340 con 0x7f1ab0109320 2024-04-14T15:29:13.889 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a9c004630 con 0x7f1ab0109320 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833313873 0 0) 0x7f1ab01a6f00 con 0x7f1ab0106580 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ab01a9340 con 0x7f1ab0106580 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376782033 0 0) 0x7f1ab01a8100 con 0x7f1ab010cf20 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ab01a6f00 con 0x7f1ab010cf20 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2498109639 0 0) 0x7f1ab01a9340 con 0x7f1ab0106580 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1ab01a8100 con 0x7f1ab0106580 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f1a980012d0 con 0x7f1ab0106580 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1509198911 0 0) 0x7f1a9c004630 con 0x7f1ab0109320 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1ab01a9340 con 0x7f1ab0109320 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f1aa0001830 con 0x7f1ab0109320 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3795465438 0 0) 0x7f1ab01a6f00 con 0x7f1ab010cf20 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1a9c004630 con 0x7f1ab010cf20 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.887+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f1aa80047d0 con 0x7f1ab010cf20 2024-04-14T15:29:13.890 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1944821421 0 0) 0x7f1ab01a8100 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 >> v1:172.21.15.45:6790/0 conn(0x7f1ab0109320 legacy=0x7f1ab0102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 >> v1:172.21.15.71:6789/0 conn(0x7f1ab010cf20 legacy=0x7f1ab01a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab01aa580 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a9570 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a9ac0 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f1a98001530 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.888+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f1a98005b10 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.889+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a90005080 con 0x7f1ab0106580 2024-04-14T15:29:13.891 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.889+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f1a9801c6c0 con 0x7f1ab0106580 2024-04-14T15:29:13.892 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.890+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(769..769 src has 1..769) ==== 8174+0+0 (unknown 4023930279 0 0) 0x7f1a9808aa10 con 0x7f1ab0106580 2024-04-14T15:29:13.892 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.890+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=770}) -- 0x7f1ab01a8100 con 0x7f1ab0106580 2024-04-14T15:29:13.894 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:13.894+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f1a98057480 con 0x7f1ab0106580 2024-04-14T15:29:14.081 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:14.081+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"} v 0) -- 0x7f1a90005370 con 0x7f1ab0106580 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: pgmap v1255: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: osdmap e768: 8 total, 8 up, 8 in 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:13 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: pgmap v1255: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: osdmap e768: 8 total, 8 up, 8 in 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: pgmap v1255: 188 pgs: 188 active+clean; 859 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: osdmap e768: 8 total, 8 up, 8 in 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1826922947' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: from='client.57179 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]: dispatch 2024-04-14T15:29:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:13 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:14.788 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:14.789+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v770) ==== 221+0+0 (unknown 1196469275 0 0) 0x7f1a9805b130 con 0x7f1ab0106580 2024-04-14T15:29:14.789 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:14.789+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 11 ==== osd_map(770..770 src has 1..770) ==== 628+0+0 (unknown 1559780301 0 0) 0x7f1a980886b0 con 0x7f1ab0106580 2024-04-14T15:29:14.789 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:14.789+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=771}) -- 0x7f1ab01a6f00 con 0x7f1ab0106580 2024-04-14T15:29:14.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:29:14 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:29:14.911 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:14.911+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"} v 0) -- 0x7f1a90000c40 con 0x7f1ab0106580 2024-04-14T15:29:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:14 smithi071 ceph-mon[33150]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:14 smithi071 ceph-mon[33150]: osdmap e769: 8 total, 8 up, 8 in 2024-04-14T15:29:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:14 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:15.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:14 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:15.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[37795]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:15.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[37795]: osdmap e769: 8 total, 8 up, 8 in 2024-04-14T15:29:15.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:15.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:15.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[29624]: from='client.57179 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_bytes", "val": "0"}]': finished 2024-04-14T15:29:15.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[29624]: osdmap e769: 8 total, 8 up, 8 in 2024-04-14T15:29:15.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:15.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:14 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:15.804 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.803+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 v771) ==== 221+0+0 (unknown 3338698234 0 0) 0x7f1a9805f9f0 con 0x7f1ab0106580 2024-04-14T15:29:15.804 INFO:tasks.workunit.client.0.smithi045.stderr:set-quota max_objects = 0 for pool c485d52f-0910-43c8-a4a6-e912c4af97a1 2024-04-14T15:29:15.804 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.804+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 <== mon.0 v1:172.21.15.45:6789/0 13 ==== osd_map(771..771 src has 1..771) ==== 628+0+0 (unknown 1420127266 0 0) 0x7f1a980533e0 con 0x7f1ab0106580 2024-04-14T15:29:15.804 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.804+0000 7f1aacff9640 1 -- 172.21.15.45:0/1386457216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=772}) -- 0x7f1a9c07d5c0 con 0x7f1ab0106580 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 >> v1:172.21.15.45:6800/1718216366 conn(0x7f1a9c06fa20 legacy=0x7f1a9c071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 >> v1:172.21.15.45:6789/0 conn(0x7f1ab0106580 legacy=0x7f1ab0102100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 shutdown_connections 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 >> 172.21.15.45:0/1386457216 conn(0x7f1ab0077150 msgr2=0x7f1ab010f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 shutdown_connections 2024-04-14T15:29:15.808 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:15.808+0000 7f1ab5f1e640 1 -- 172.21.15.45:0/1386457216 wait complete. 2024-04-14T15:29:15.821 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 30 2024-04-14T15:29:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:15 smithi071 ceph-mon[33150]: pgmap v1258: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-04-14T15:29:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:15 smithi071 ceph-mon[33150]: osdmap e770: 8 total, 8 up, 8 in 2024-04-14T15:29:16.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:15 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[37795]: pgmap v1258: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-04-14T15:29:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[37795]: osdmap e770: 8 total, 8 up, 8 in 2024-04-14T15:29:16.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[29624]: pgmap v1258: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-04-14T15:29:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[29624]: osdmap e770: 8 total, 8 up, 8 in 2024-04-14T15:29:16.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:15 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]: dispatch 2024-04-14T15:29:16.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:16 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:29:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:29:16.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:16 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:16.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:16 smithi045 ceph-mon[37795]: osdmap e771: 8 total, 8 up, 8 in 2024-04-14T15:29:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:16 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:16.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:16 smithi045 ceph-mon[29624]: osdmap e771: 8 total, 8 up, 8 in 2024-04-14T15:29:17.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:16 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1386457216' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "field": "max_objects", "val": "0"}]': finished 2024-04-14T15:29:17.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:16 smithi071 ceph-mon[33150]: osdmap e771: 8 total, 8 up, 8 in 2024-04-14T15:29:18.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:17 smithi071 ceph-mon[33150]: pgmap v1261: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-04-14T15:29:18.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:17 smithi045 ceph-mon[37795]: pgmap v1261: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-04-14T15:29:18.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:17 smithi045 ceph-mon[29624]: pgmap v1261: 188 pgs: 188 active+clean; 871 KiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-04-14T15:29:20.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:19 smithi071 ceph-mon[33150]: pgmap v1262: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:29:20.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:19 smithi045 ceph-mon[29624]: pgmap v1262: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:29:20.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:19 smithi045 ceph-mon[37795]: pgmap v1262: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-04-14T15:29:22.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:21 smithi071 ceph-mon[33150]: pgmap v1263: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 0 op/s 2024-04-14T15:29:22.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:21 smithi045 ceph-mon[29624]: pgmap v1263: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 0 op/s 2024-04-14T15:29:22.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:21 smithi045 ceph-mon[37795]: pgmap v1263: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 739 B/s rd, 0 op/s 2024-04-14T15:29:24.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:23 smithi071 ceph-mon[33150]: pgmap v1264: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:29:24.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:23 smithi045 ceph-mon[29624]: pgmap v1264: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:29:24.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:23 smithi045 ceph-mon[37795]: pgmap v1264: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-04-14T15:29:24.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:29:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:29:25.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:24 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:25.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:24 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:25.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:24 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:26.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:25 smithi071 ceph-mon[33150]: pgmap v1265: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:29:26.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:25 smithi045 ceph-mon[29624]: pgmap v1265: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:29:26.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:25 smithi045 ceph-mon[37795]: pgmap v1265: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-14T15:29:26.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:26 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:29:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:29:28.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:27 smithi071 ceph-mon[33150]: pgmap v1266: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-04-14T15:29:28.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:27 smithi045 ceph-mon[29624]: pgmap v1266: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-04-14T15:29:28.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:27 smithi045 ceph-mon[37795]: pgmap v1266: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-04-14T15:29:29.106 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:28 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:29.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:28 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:29.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:28 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:30.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:29 smithi071 ceph-mon[33150]: pgmap v1267: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:30.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:29 smithi045 ceph-mon[29624]: pgmap v1267: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:30.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:29 smithi045 ceph-mon[37795]: pgmap v1267: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:31.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:30 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:29:31.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:30 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:29:31.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:30 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-14T15:29:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:31 smithi071 ceph-mon[33150]: pgmap v1268: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:31 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:29:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:31 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:29:32.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:31 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[37795]: pgmap v1268: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[29624]: pgmap v1268: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-14T15:29:32.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:31 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' 2024-04-14T15:29:34.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:33 smithi071 ceph-mon[33150]: pgmap v1269: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:34.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:33 smithi045 ceph-mon[37795]: pgmap v1269: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:34.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:33 smithi045 ceph-mon[29624]: pgmap v1269: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:34.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:29:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:29:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:35 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:36.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:35 smithi071 ceph-mon[33150]: pgmap v1270: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:36.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:35 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:36.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:35 smithi045 ceph-mon[29624]: pgmap v1270: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:36.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:35 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:36.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:35 smithi045 ceph-mon[37795]: pgmap v1270: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:36.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:36 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:29:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:29:38.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:37 smithi071 ceph-mon[33150]: pgmap v1271: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:38.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:37 smithi045 ceph-mon[37795]: pgmap v1271: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:38.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:37 smithi045 ceph-mon[29624]: pgmap v1271: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:40.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:39 smithi071 ceph-mon[33150]: pgmap v1272: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:40.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:39 smithi045 ceph-mon[37795]: pgmap v1272: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:40.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:39 smithi045 ceph-mon[29624]: pgmap v1272: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:42.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:41 smithi071 ceph-mon[33150]: pgmap v1273: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:42.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:41 smithi045 ceph-mon[37795]: pgmap v1273: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:42.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:41 smithi045 ceph-mon[29624]: pgmap v1273: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:43 smithi071 ceph-mon[33150]: pgmap v1274: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:44.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:43 smithi071 ceph-mon[33150]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:43 smithi045 ceph-mon[29624]: pgmap v1274: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:44.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:43 smithi045 ceph-mon[29624]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:43 smithi045 ceph-mon[37795]: pgmap v1274: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:44.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:43 smithi045 ceph-mon[37795]: from='mgr.14610 v1:172.21.15.45:0/1940931494' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-14T15:29:44.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:29:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug there is no tcmu-runner data available 2024-04-14T15:29:45.823 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool delete c485d52f-0910-43c8-a4a6-e912c4af97a1 c485d52f-0910-43c8-a4a6-e912c4af97a1 --yes-i-really-really-mean-it 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.916+0000 7f6e3951e640 1 -- 172.21.15.45:0/2915306039 >> v1:172.21.15.45:6789/0 conn(0x7f6e341020a0 legacy=0x7f6e34102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.916+0000 7f6e3951e640 1 -- 172.21.15.45:0/2915306039 shutdown_connections 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.916+0000 7f6e3951e640 1 -- 172.21.15.45:0/2915306039 >> 172.21.15.45:0/2915306039 conn(0x7f6e340fd8e0 msgr2=0x7f6e340ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.916+0000 7f6e3951e640 1 -- 172.21.15.45:0/2915306039 shutdown_connections 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 -- 172.21.15.45:0/2915306039 wait complete. 2024-04-14T15:29:45.918 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 Processor -- start 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 -- start start 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e341a6e90 con 0x7f6e3410ad20 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e341a80d0 con 0x7f6e341020a0 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e3951e640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7f6e341a9310 con 0x7f6e3410e890 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e2bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6e3410ad20 0x7f6e341a1d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34974/0 (socket says 172.21.15.45:34974) 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e337fe640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7f6e3410e890 0x7f6e341a5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:36388/0 (socket says 172.21.15.45:36388) 2024-04-14T15:29:45.919 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.917+0000 7f6e2bfff640 1 -- 172.21.15.45:0/2973213573 learned_addr learned my addr 172.21.15.45:0/2973213573 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:29:45.921 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590650180 0 0) 0x7f6e341a6e90 con 0x7f6e3410ad20 2024-04-14T15:29:45.921 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e20003160 con 0x7f6e3410ad20 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3039379411 0 0) 0x7f6e341a9310 con 0x7f6e3410e890 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e341a6e90 con 0x7f6e3410e890 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46160673 0 0) 0x7f6e341a80d0 con 0x7f6e341020a0 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e341a9310 con 0x7f6e341020a0 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1120413310 0 0) 0x7f6e20003160 con 0x7f6e3410ad20 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e341a80d0 con 0x7f6e3410ad20 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6e1c0013d0 con 0x7f6e3410ad20 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 598525556 0 0) 0x7f6e341a6e90 con 0x7f6e3410e890 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e20003160 con 0x7f6e3410e890 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.1 v1:172.21.15.71:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2490936497 0 0) 0x7f6e341a9310 con 0x7f6e341020a0 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.71:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6e341a6e90 con 0x7f6e341020a0 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6e2c001bb0 con 0x7f6e3410e890 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.1 v1:172.21.15.71:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6e24001980 con 0x7f6e341020a0 2024-04-14T15:29:45.922 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2490567806 0 0) 0x7f6e341a80d0 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 >> v1:172.21.15.45:6790/0 conn(0x7f6e3410e890 legacy=0x7f6e341a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 >> v1:172.21.15.71:6789/0 conn(0x7f6e341020a0 legacy=0x7f6e341073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e341aa550 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6e341a9540 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6e341a9a10 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7f6e1c001570 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.918+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7f6e1c005bd0 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.919+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e14005080 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.920+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7f6e1c0038a0 con 0x7f6e3410ad20 2024-04-14T15:29:45.923 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.920+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 8 ==== osd_map(771..771 src has 1..771) ==== 8174+0+0 (unknown 2314381485 0 0) 0x7f6e1c08a830 con 0x7f6e3410ad20 2024-04-14T15:29:45.924 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.920+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=772}) -- 0x7f6e341a80d0 con 0x7f6e3410ad20 2024-04-14T15:29:45.926 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:45.925+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7f6e1c057240 con 0x7f6e3410ad20 2024-04-14T15:29:46.114 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.113+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e14005370 con 0x7f6e3410ad20 2024-04-14T15:29:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:45 smithi045 ceph-mon[37795]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:46.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:45 smithi045 ceph-mon[37795]: pgmap v1275: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:45 smithi045 ceph-mon[29624]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:46.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:45 smithi045 ceph-mon[29624]: pgmap v1275: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:45 smithi071 ceph-mon[33150]: from='client.24463 v1:172.21.15.71:0/2483469111' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-04-14T15:29:46.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:45 smithi071 ceph-mon[33150]: pgmap v1275: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-14T15:29:46.865 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.864+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]=0 pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' removed v772) ==== 248+0+0 (unknown 2072377538 0 0) 0x7f6e1c05aef0 con 0x7f6e3410ad20 2024-04-14T15:29:46.866 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.865+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 11 ==== osd_map(772..772 src has 1..772) ==== 296+0+0 (unknown 599443985 0 0) 0x7f6e1c088470 con 0x7f6e3410ad20 2024-04-14T15:29:46.866 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.865+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_subscribe({osdmap=773}) -- 0x7f6e341a6e90 con 0x7f6e3410ad20 2024-04-14T15:29:46.991 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.990+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 --> v1:172.21.15.45:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6e14005da0 con 0x7f6e3410ad20 2024-04-14T15:29:46.991 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.990+0000 7f6e30ff9640 1 -- 172.21.15.45:0/2973213573 <== mon.0 v1:172.21.15.45:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]=0 pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' does not exist v772) ==== 255+0+0 (unknown 1354418107 0 0) 0x7f6e1c05f7b0 con 0x7f6e3410ad20 2024-04-14T15:29:46.992 INFO:tasks.workunit.client.0.smithi045.stderr:pool 'c485d52f-0910-43c8-a4a6-e912c4af97a1' does not exist 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 >> v1:172.21.15.45:6800/1718216366 conn(0x7f6e2006f960 legacy=0x7f6e20071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 >> v1:172.21.15.45:6789/0 conn(0x7f6e3410ad20 legacy=0x7f6e341a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 shutdown_connections 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 >> 172.21.15.45:0/2973213573 conn(0x7f6e340fd8e0 msgr2=0x7f6e340ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 shutdown_connections 2024-04-14T15:29:46.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:46.993+0000 7f6e3951e640 1 -- 172.21.15.45:0/2973213573 wait complete. 2024-04-14T15:29:46.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:46 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:46.996 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:46 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: ::ffff:172.21.15.71 - - [14/Apr/2024:15:29:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-04-14T15:29:46.996 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:46 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:47.003 INFO:tasks.workunit.client.0.smithi045.stderr:+ ceph osd pool delete aac2e7fb-7118-4811-98cd-84307849713f aac2e7fb-7118-4811-98cd-84307849713f --yes-i-really-really-mean-it 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.085+0000 7fda9055e640 1 -- 172.21.15.45:0/682327495 >> v1:172.21.15.45:6790/0 conn(0x7fda88103010 legacy=0x7fda881033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- 172.21.15.45:0/682327495 shutdown_connections 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- 172.21.15.45:0/682327495 >> 172.21.15.45:0/682327495 conn(0x7fda880fc850 msgr2=0x7fda880fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- 172.21.15.45:0/682327495 shutdown_connections 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- 172.21.15.45:0/682327495 wait complete. 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 Processor -- start 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- start start 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda881a92c0 con 0x7fda8810ae10 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- --> v1:172.21.15.71:6789/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda881aa4a0 con 0x7fda88103010 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda9055e640 1 -- --> v1:172.21.15.45:6790/0 -- auth(proto 0 34 bytes epoch 0) -- 0x7fda881ab680 con 0x7fda8810e920 2024-04-14T15:29:47.089 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda8eafb640 1 --1- >> v1:172.21.15.45:6790/0 conn(0x7fda8810e920 0x7fda881a7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.45:6790/0 says I am v1:172.21.15.45:36396/0 (socket says 172.21.15.45:36396) 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.086+0000 7fda8eafb640 1 -- 172.21.15.45:0/1841852750 learned_addr learned my addr 172.21.15.45:0/1841852750 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3436328498 0 0) 0x7fda881ab680 con 0x7fda8810e920 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda74003160 con 0x7fda8810e920 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1928495272 0 0) 0x7fda881a92c0 con 0x7fda8810ae10 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda881ab680 con 0x7fda8810ae10 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.1 v1:172.21.15.71:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 114457286 0 0) 0x7fda881aa4a0 con 0x7fda88103010 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.71:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda881a92c0 con 0x7fda88103010 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1468167769 0 0) 0x7fda74003160 con 0x7fda8810e920 2024-04-14T15:29:47.090 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fda881aa4a0 con 0x7fda8810e920 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fda80001bb0 con 0x7fda8810e920 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 836274763 0 0) 0x7fda881ab680 con 0x7fda8810ae10 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fda74003160 con 0x7fda8810ae10 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fda780013b0 con 0x7fda8810ae10 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3398118616 0 0) 0x7fda881aa4a0 con 0x7fda8810e920 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 >> v1:172.21.15.71:6789/0 conn(0x7fda88103010 legacy=0x7fda8819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:47.091 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 >> v1:172.21.15.45:6789/0 conn(0x7fda8810ae10 legacy=0x7fda881a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:47.092 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881ac860 con 0x7fda8810e920 2024-04-14T15:29:47.092 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fda881a94f0 con 0x7fda8810e920 2024-04-14T15:29:47.092 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fda881a99c0 con 0x7fda8810e920 2024-04-14T15:29:47.092 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 5 ==== config(25 keys) ==== 1068+0+0 (unknown 887181305 0 0) 0x7fda80003e20 con 0x7fda8810e920 2024-04-14T15:29:47.092 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.087+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 6 ==== mon_map magic: 0 ==== 308+0+0 (unknown 2097602899 0 0) 0x7fda80005f20 con 0x7fda8810e920 2024-04-14T15:29:47.093 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.088+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c005080 con 0x7fda8810e920 2024-04-14T15:29:47.093 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.089+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 7 ==== mgrmap(e 22) ==== 92680+0+0 (unknown 3644435065 0 0) 0x7fda8001c930 con 0x7fda8810e920 2024-04-14T15:29:47.093 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.089+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 8 ==== osd_map(772..772 src has 1..772) ==== 7786+0+0 (unknown 3342481854 0 0) 0x7fda8008a920 con 0x7fda8810e920 2024-04-14T15:29:47.093 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.089+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=773}) -- 0x7fda881aa4a0 con 0x7fda8810e920 2024-04-14T15:29:47.094 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.094+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188297 (unknown 1092875540 0 2267177403) 0x7fda8005fb60 con 0x7fda8810e920 2024-04-14T15:29:47.280 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.279+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda6c005370 con 0x7fda8810e920 2024-04-14T15:29:47.356 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:46 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:47.867 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.866+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 10 ==== osd_map(773..773 src has 1..773) ==== 296+0+0 (unknown 609693907 0 0) 0x7fda80052d70 con 0x7fda8810e920 2024-04-14T15:29:47.868 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.866+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_subscribe({osdmap=774}) -- 0x7fda881a92c0 con 0x7fda8810e920 2024-04-14T15:29:47.871 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.870+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]=0 pool 'aac2e7fb-7118-4811-98cd-84307849713f' removed v773) ==== 248+0+0 (unknown 2922186182 0 0) 0x7fda80053690 con 0x7fda8810e920 2024-04-14T15:29:47.991 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.990+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 --> v1:172.21.15.45:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true} v 0) -- 0x7fda6c005da0 con 0x7fda8810e920 2024-04-14T15:29:47.991 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.990+0000 7fda7f7fe640 1 -- 172.21.15.45:0/1841852750 <== mon.2 v1:172.21.15.45:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]=0 pool 'aac2e7fb-7118-4811-98cd-84307849713f' does not exist v773) ==== 255+0+0 (unknown 2363202930 0 0) 0x7fda800530f0 con 0x7fda8810e920 2024-04-14T15:29:47.991 INFO:tasks.workunit.client.0.smithi045.stderr:pool 'aac2e7fb-7118-4811-98cd-84307849713f' does not exist 2024-04-14T15:29:47.993 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.992+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 >> v1:172.21.15.45:6800/1718216366 conn(0x7fda7406f670 legacy=0x7fda74071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:47.993 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.992+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 >> v1:172.21.15.45:6790/0 conn(0x7fda8810e920 legacy=0x7fda881a7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-14T15:29:47.993 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.992+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 shutdown_connections 2024-04-14T15:29:47.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.992+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 >> 172.21.15.45:0/1841852750 conn(0x7fda880fc850 msgr2=0x7fda88110db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-14T15:29:47.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.993+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 shutdown_connections 2024-04-14T15:29:47.994 INFO:tasks.workunit.client.0.smithi045.stderr:2024-04-14T15:29:47.993+0000 7fda9055e640 1 -- 172.21.15.45:0/1841852750 wait complete. 2024-04-14T15:29:48.002 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo OK 2024-04-14T15:29:48.003 INFO:tasks.workunit.client.0.smithi045.stdout:OK 2024-04-14T15:29:48.004 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-14T15:29:48.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-14T15:29:48.032 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-04-14T15:29:48.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: pgmap v1276: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: osdmap e772: 8 total, 8 up, 8 in 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[29624]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: pgmap v1276: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: osdmap e772: 8 total, 8 up, 8 in 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.247 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:47 smithi045 ceph-mon[37795]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: pgmap v1276: 188 pgs: 188 active+clean; 871 KiB data, 815 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: osdmap e772: 8 total, 8 up, 8 in 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/2973213573' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "pool2": "c485d52f-0910-43c8-a4a6-e912c4af97a1", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:47 smithi071 ceph-mon[33150]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.379 DEBUG:teuthology.parallel:result is None 2024-04-14T15:29:48.379 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-14T15:29:48.406 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-14T15:29:48.406 DEBUG:teuthology.orchestra.run.smithi045:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-14T15:29:48.504 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-14T15:29:48.504 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-14T15:29:48.516 INFO:tasks.cephadm:Teardown begin 2024-04-14T15:29:48.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:29:48.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:29:48.580 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-14T15:29:48.580 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-14T15:29:48.600 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-14T15:29:48.636 INFO:tasks.cephadm:Stopping all daemons... 2024-04-14T15:29:48.637 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-14T15:29:48.637 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a 2024-04-14T15:29:48.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[37795]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:29:48.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[37795]: from='client.57197 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:48.996 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[37795]: osdmap e773: 8 total, 8 up, 8 in 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[37795]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[37795]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 systemd[1]: Stopping Ceph mon.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[29624]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[29624]: from='client.57197 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[29624]: osdmap e773: 8 total, 8 up, 8 in 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[29624]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:48.997 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:48 smithi045 ceph-mon[29624]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:48 smithi071 ceph-mon[33150]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-04-14T15:29:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:48 smithi071 ceph-mon[33150]: from='client.57197 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]': finished 2024-04-14T15:29:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:48 smithi071 ceph-mon[33150]: osdmap e773: 8 total, 8 up, 8 in 2024-04-14T15:29:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:48 smithi071 ceph-mon[33150]: from='client.? v1:172.21.15.45:0/1841852750' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:49.357 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:48 smithi071 ceph-mon[33150]: from='client.57197 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "aac2e7fb-7118-4811-98cd-84307849713f", "pool2": "aac2e7fb-7118-4811-98cd-84307849713f", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a[29619]: 2024-04-14T15:29:49.075+0000 7f30835bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a[29619]: 2024-04-14T15:29:49.075+0000 7f30835bf700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:49 smithi045 podman[146460]: 2024-04-14 15:29:49.158629179 +0000 UTC m=+0.331696724 container died 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE Bus STOPPING 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE Bus STOPPED 2024-04-14T15:29:49.424 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE Bus STARTING 2024-04-14T15:29:49.425 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE Serving on http://:::9283 2024-04-14T15:29:49.425 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:49 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:49] ENGINE Bus STARTED 2024-04-14T15:29:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:49 smithi045 podman[146460]: 2024-04-14 15:29:49.632888861 +0000 UTC m=+0.805956413 container cleanup 5e29a662536471c4b996d41f14b6663646d8356e5d884d60852ad190faaf4933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a, ceph=True, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-14T15:29:49.746 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 14 15:29:49 smithi045 bash[146460]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-a 2024-04-14T15:29:50.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.a.service' 2024-04-14T15:29:50.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:29:50.103 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-14T15:29:50.103 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-04-14T15:29:50.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.c 2024-04-14T15:29:50.496 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:50 smithi045 systemd[1]: Stopping Ceph mon.c for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:50.987 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c[37791]: 2024-04-14T15:29:50.535+0000 7fc0744bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c[37791]: 2024-04-14T15:29:50.535+0000 7fc0744bf700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:50 smithi045 podman[146595]: 2024-04-14 15:29:50.700789217 +0000 UTC m=+0.374402673 container died 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE Bus STOPPING 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE Bus STOPPED 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE Bus STARTING 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE Serving on http://:::9283 2024-04-14T15:29:50.988 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:50 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: [14/Apr/2024:15:29:50] ENGINE Bus STARTED 2024-04-14T15:29:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:51 smithi045 podman[146595]: 2024-04-14 15:29:51.079120827 +0000 UTC m=+0.752734283 container cleanup 0bb562ebd0194accbf2dd87ab14ac202d4e592b10a1e56bc0ab82e78657cf4c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-14T15:29:51.246 INFO:journalctl@ceph.mon.c.smithi045.stdout:Apr 14 15:29:51 smithi045 bash[146595]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-c 2024-04-14T15:29:51.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.c.service' 2024-04-14T15:29:51.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:29:51.497 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-04-14T15:29:51.497 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-04-14T15:29:51.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.b 2024-04-14T15:29:51.816 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:51 smithi071 systemd[1]: Stopping Ceph mon.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b[33133]: 2024-04-14T15:29:51.815+0000 7f7f3f586700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:29:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b[33133]: 2024-04-14T15:29:51.815+0000 7f7f3f586700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-04-14T15:29:52.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:52 smithi071 podman[69406]: 2024-04-14 15:29:52.020273172 +0000 UTC m=+0.329537529 container died fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-14T15:29:52.606 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:52 smithi071 podman[69406]: 2024-04-14 15:29:52.449959325 +0000 UTC m=+0.759223744 container cleanup fd13b792190086558104bf676ef7dba538b06f51953fd85703d1c2faf8527384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2) 2024-04-14T15:29:52.607 INFO:journalctl@ceph.mon.b.smithi071.stdout:Apr 14 15:29:52 smithi071 bash[69406]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mon-b 2024-04-14T15:29:52.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mon.b.service' 2024-04-14T15:29:52.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:29:52.913 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-04-14T15:29:52.913 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-14T15:29:52.913 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y 2024-04-14T15:29:53.200 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:52 smithi045 systemd[1]: Stopping Ceph mgr.y for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:53.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:29:53.198+0000 7f6019328700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:29:53.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y[29859]: 2024-04-14T15:29:53.198+0000 7f6019328700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-14T15:29:53.496 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:53 smithi045 podman[146731]: 2024-04-14 15:29:53.231700594 +0000 UTC m=+0.127902297 container died 2f17f04e2184d5ae197fb0cbad8c5741ffe248117f81211ba2577b29f1cf0358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-14T15:29:53.968 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:53 smithi045 podman[146731]: 2024-04-14 15:29:53.717311068 +0000 UTC m=+0.613512773 container cleanup 2f17f04e2184d5ae197fb0cbad8c5741ffe248117f81211ba2577b29f1cf0358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a971fd5f5719038c70b817bafacd6d5400b1d888, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y, org.label-schema.build-date=20240403, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5) 2024-04-14T15:29:53.969 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 14 15:29:53 smithi045 bash[146731]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-y 2024-04-14T15:29:53.969 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:29:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:29:53.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:29:54.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.y.service' 2024-04-14T15:29:54.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:29:54.175 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-14T15:29:54.175 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-14T15:29:54.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.x 2024-04-14T15:29:54.463 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:29:54 smithi071 systemd[1]: Stopping Ceph mgr.x for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:54.779 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:29:54 smithi071 podman[69531]: 2024-04-14 15:29:54.49311395 +0000 UTC m=+0.116018282 container died b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:29:55.106 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:29:54 smithi071 podman[69531]: 2024-04-14 15:29:54.853972774 +0000 UTC m=+0.476877106 container cleanup b031601cbe62da3ef824f3f48988a73fbc0fa64c452602e93d4cc74ae83b1547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-14T15:29:55.107 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Apr 14 15:29:54 smithi071 bash[69531]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-mgr-x 2024-04-14T15:29:55.282 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@mgr.x.service' 2024-04-14T15:29:55.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:29:55.321 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-14T15:29:55.321 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-14T15:29:55.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.0 2024-04-14T15:29:55.629 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:29:55 smithi045 systemd[1]: Stopping Ceph osd.0 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:29:55.996 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:29:55 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:29:55.628+0000 7f4e87cdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:29:55.996 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:29:55 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:29:55.628+0000 7f4e87cdd700 -1 osd.0 773 *** Got signal Terminated *** 2024-04-14T15:29:55.996 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:29:55 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0[41699]: 2024-04-14T15:29:55.628+0000 7f4e87cdd700 -1 osd.0 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:29:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:29:59.754Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:30:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:29:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:29:59.754Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:30:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:29:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:29:59.758Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:30:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:29:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:29:59.758Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:30:00.108 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:29:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:29:59.758Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:30:01.338 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:01 smithi045 podman[146854]: 2024-04-14 15:30:01.072975023 +0000 UTC m=+5.553356890 container died 3044a6a1146437445809fcbc2d5f7d5fe26e5dc10a6c8f9e864574ceaf2ff9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-14T15:30:01.747 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:01 smithi045 podman[146854]: 2024-04-14 15:30:01.437981117 +0000 UTC m=+5.918362987 container cleanup 3044a6a1146437445809fcbc2d5f7d5fe26e5dc10a6c8f9e864574ceaf2ff9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-14T15:30:01.747 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:01 smithi045 bash[146854]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0 2024-04-14T15:30:02.211 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:01 smithi045 podman[146872]: 2024-04-14 15:30:01.847653191 +0000 UTC m=+0.768702620 container remove 3044a6a1146437445809fcbc2d5f7d5fe26e5dc10a6c8f9e864574ceaf2ff9ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, RELEASE=main-a971fd5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-14T15:30:02.496 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.114851968 +0000 UTC m=+0.017840803 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:02.497 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.215037059 +0000 UTC m=+0.118025936 container create 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403) 2024-04-14T15:30:02.497 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.434385342 +0000 UTC m=+0.337374193 container init 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, ceph=True, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-14T15:30:02.497 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.437915717 +0000 UTC m=+0.340904556 container start 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-14T15:30:02.996 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.56789522 +0000 UTC m=+0.470884055 container attach 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, ceph=True, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-14T15:30:02.997 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:02 smithi045 podman[146964]: 2024-04-14 15:30:02.736002031 +0000 UTC m=+0.638990867 container died 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS) 2024-04-14T15:30:03.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.0.service' 2024-04-14T15:30:03.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:03 smithi045 podman[146990]: 2024-04-14 15:30:03.296099272 +0000 UTC m=+0.551575290 container remove 9a89ec10a89afb32f2e372ecd5b161a7800d3d49842687682e801a7c9ebdbdd1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-14T15:30:03.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:03 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-14T15:30:03.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:03 smithi045 systemd[1]: Stopped Ceph osd.0 for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:30:03.317 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 14 15:30:03 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.0.service: Consumed 40.538s CPU time. 2024-04-14T15:30:03.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:03.351 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-14T15:30:03.352 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-14T15:30:03.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.1 2024-04-14T15:30:03.571 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:03 smithi045 systemd[1]: Stopping Ceph osd.1 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:03.824 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:03.571Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:03.824 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:03.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:04.246 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:30:03.824+0000 7ff0c4f1e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:04.246 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:30:03.824+0000 7ff0c4f1e700 -1 osd.1 773 *** Got signal Terminated *** 2024-04-14T15:30:04.246 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1[47646]: 2024-04-14T15:30:03.824+0000 7ff0c4f1e700 -1 osd.1 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:09.497 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:09 smithi045 podman[147070]: 2024-04-14 15:30:09.180025363 +0000 UTC m=+5.596083345 container died cca6c753829facc8c871d7a56426fd286112cae5ccbf4a76dd22d93450166d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-a971fd5) 2024-04-14T15:30:09.925 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:09 smithi045 podman[147070]: 2024-04-14 15:30:09.691039767 +0000 UTC m=+6.107097788 container cleanup cca6c753829facc8c871d7a56426fd286112cae5ccbf4a76dd22d93450166d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-14T15:30:09.925 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:09 smithi045 bash[147070]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1 2024-04-14T15:30:10.246 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:09 smithi045 podman[147096]: 2024-04-14 15:30:09.925648288 +0000 UTC m=+0.739582192 container remove cca6c753829facc8c871d7a56426fd286112cae5ccbf4a76dd22d93450166d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-14T15:30:10.605 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.259955421 +0000 UTC m=+0.111076368 container create 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-14T15:30:10.605 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.166356447 +0000 UTC m=+0.017477367 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:10.605 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.503695106 +0000 UTC m=+0.354816015 container init 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5) 2024-04-14T15:30:10.605 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.506820185 +0000 UTC m=+0.357941098 container start 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-14T15:30:10.605 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.6037723 +0000 UTC m=+0.454893208 container attach 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-a971fd5, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403) 2024-04-14T15:30:10.996 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:10 smithi045 podman[147192]: 2024-04-14 15:30:10.803843845 +0000 UTC m=+0.654964747 container died 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-14T15:30:11.489 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 14 15:30:11 smithi045 podman[147218]: 2024-04-14 15:30:11.465304092 +0000 UTC m=+0.656253941 container remove 9f8b08c13e1e327c7747ce7837005fd4c48b000cf5562845e0d5f383bbdf47eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:11.493 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.1.service' 2024-04-14T15:30:11.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:11.538 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-14T15:30:11.538 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-14T15:30:11.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.2 2024-04-14T15:30:11.746 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:11 smithi045 systemd[1]: Stopping Ceph osd.2 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:12.246 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2[53733]: 2024-04-14T15:30:11.876+0000 7fe4387bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:12.246 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2[53733]: 2024-04-14T15:30:11.876+0000 7fe4387bc700 -1 osd.2 773 *** Got signal Terminated *** 2024-04-14T15:30:12.246 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:11 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2[53733]: 2024-04-14T15:30:11.876+0000 7fe4387bc700 -1 osd.2 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:13.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:13 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:13.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:13.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:13 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:13.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:13.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:13 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:13.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:17.460 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:17 smithi045 podman[147298]: 2024-04-14 15:30:17.207906303 +0000 UTC m=+5.445421921 container died e925397c0f16835649c54553cba369b6121a905757448f2b7272077dfa9d4c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-14T15:30:17.746 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:17 smithi045 podman[147298]: 2024-04-14 15:30:17.668017807 +0000 UTC m=+5.905533425 container cleanup e925397c0f16835649c54553cba369b6121a905757448f2b7272077dfa9d4c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:17.746 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:17 smithi045 bash[147298]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2 2024-04-14T15:30:18.235 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:17 smithi045 podman[147318]: 2024-04-14 15:30:17.919012041 +0000 UTC m=+0.713343653 container remove e925397c0f16835649c54553cba369b6121a905757448f2b7272077dfa9d4c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-14T15:30:18.496 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.236372294 +0000 UTC m=+0.109808580 container create 9f9f731d0d68e8202af61a975c63a51d231a3b5c277902aad4c446bc1c0f9380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:18.496 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.14412977 +0000 UTC m=+0.017566078 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:18.496 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.472126743 +0000 UTC m=+0.345563028 container init 9f9f731d0d68e8202af61a975c63a51d231a3b5c277902aad4c446bc1c0f9380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:18.496 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.475205806 +0000 UTC m=+0.348642092 container start 9f9f731d0d68e8202af61a975c63a51d231a3b5c277902aad4c446bc1c0f9380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:18.996 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.546938202 +0000 UTC m=+0.420374504 container attach 9f9f731d0d68e8202af61a975c63a51d231a3b5c277902aad4c446bc1c0f9380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-14T15:30:18.997 INFO:journalctl@ceph.osd.2.smithi045.stdout:Apr 14 15:30:18 smithi045 podman[147413]: 2024-04-14 15:30:18.775372052 +0000 UTC m=+0.648808331 container died 9f9f731d0d68e8202af61a975c63a51d231a3b5c277902aad4c446bc1c0f9380 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-14T15:30:19.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.2.service' 2024-04-14T15:30:19.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:19.502 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-14T15:30:19.502 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-14T15:30:19.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.3 2024-04-14T15:30:19.746 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:19 smithi045 systemd[1]: Stopping Ceph osd.3 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:20.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:19 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:19.836+0000 7fb7e4275700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:20.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:19 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:19.836+0000 7fb7e4275700 -1 osd.3 773 *** Got signal Terminated *** 2024-04-14T15:30:20.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:19 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:19.836+0000 7fb7e4275700 -1 osd.3 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:22.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:21 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:21.750+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:22.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:21 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:21.935+0000 7fb7e0869700 -1 osd.3 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:56.386723+0000 front 2024-04-14T15:29:56.386757+0000 (oldest deadline 2024-04-14T15:30:21.686756+0000) 2024-04-14T15:30:23.008 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:22 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:22.730+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:23.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:22 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:22.985+0000 7fb7e0869700 -1 osd.3 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:56.386723+0000 front 2024-04-14T15:29:56.386757+0000 (oldest deadline 2024-04-14T15:30:21.686756+0000) 2024-04-14T15:30:23.992 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:23.573Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:23.992 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:23.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:23.992 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:23.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:24.080 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:23.692+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:24.080 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:23 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:23.827+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:24.246 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:23 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3[59443]: 2024-04-14T15:30:23.992+0000 7fb7e0869700 -1 osd.3 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:56.386723+0000 front 2024-04-14T15:29:56.386757+0000 (oldest deadline 2024-04-14T15:30:21.686756+0000) 2024-04-14T15:30:24.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:24.079+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:24.695 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:24.428+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:25.054 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:24.865+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:25.054 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:24 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:24.694+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:25.055 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:25 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:25.054+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:25.419 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:25 smithi045 podman[147521]: 2024-04-14 15:30:25.147336453 +0000 UTC m=+5.412681842 container died 5ca3ff6f9ff0b701d43bbf6200a37c7a82d4de7a14ebebd6bc8ed51c886c5839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-14T15:30:25.747 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:25 smithi045 podman[147521]: 2024-04-14 15:30:25.511064219 +0000 UTC m=+5.776409640 container cleanup 5ca3ff6f9ff0b701d43bbf6200a37c7a82d4de7a14ebebd6bc8ed51c886c5839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:25.747 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:25 smithi045 bash[147521]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3 2024-04-14T15:30:25.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:25 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:25.460+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:25.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:25 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:25.699+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:26.129 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:25 smithi045 podman[147539]: 2024-04-14 15:30:25.79559796 +0000 UTC m=+0.641893405 container remove 5ca3ff6f9ff0b701d43bbf6200a37c7a82d4de7a14ebebd6bc8ed51c886c5839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:26.357 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:25 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:25.892+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:26.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:26 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:26.041+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:26.465 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.032201773 +0000 UTC m=+0.017623205 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:26.465 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.137937471 +0000 UTC m=+0.123358930 container create cadad58303bb91402a54818c3afe2f59af2cf8f132bb9b798e5705a8430abfd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS) 2024-04-14T15:30:26.746 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.465397134 +0000 UTC m=+0.450818561 container init cadad58303bb91402a54818c3afe2f59af2cf8f132bb9b798e5705a8430abfd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:26.746 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.468777833 +0000 UTC m=+0.454199251 container start cadad58303bb91402a54818c3afe2f59af2cf8f132bb9b798e5705a8430abfd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-14T15:30:26.746 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.56542347 +0000 UTC m=+0.550844901 container attach cadad58303bb91402a54818c3afe2f59af2cf8f132bb9b798e5705a8430abfd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-14T15:30:26.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:26 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:26.488+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:26.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:26 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:26.695+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:27.050 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 14 15:30:26 smithi045 podman[147632]: 2024-04-14 15:30:26.763862599 +0000 UTC m=+0.749284020 container died cadad58303bb91402a54818c3afe2f59af2cf8f132bb9b798e5705a8430abfd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-14T15:30:27.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.3.service' 2024-04-14T15:30:27.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:27.306 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-14T15:30:27.306 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-14T15:30:27.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.4 2024-04-14T15:30:27.337 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:26 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:26.907+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:27.338 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:27.016+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:27.607 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:27 smithi071 systemd[1]: Stopping Ceph osd.4 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:27.607 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:27.447+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:28.051 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:27.700+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:28.052 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:27.786+0000 7f2724145700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:28.052 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:27.786+0000 7f2724145700 -1 osd.4 773 *** Got signal Terminated *** 2024-04-14T15:30:28.052 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:27.786+0000 7f2724145700 -1 osd.4 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:28.052 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:27 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:27.859+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:28.356 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:28 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:28.050+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:28.748 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:28 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:28.437+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:29.072 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:28 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:28.819+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:29.072 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:28 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:28.747+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:29.356 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:29 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:29.071+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:29.777 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:29 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:29.468+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:30.079 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:29 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:29.783+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:30.079 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:29 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:29.776+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:30.079 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:29 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:29.776+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:30.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:30.078+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:30.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:30.078+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:30.776 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:30.429+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:31.097 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:30.774+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:31.098 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:30.774+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:06.393821+0000 front 2024-04-14T15:30:06.393873+0000 (oldest deadline 2024-04-14T15:30:30.493668+0000) 2024-04-14T15:30:31.098 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:30.823+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:31.098 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:30.823+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:31.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:31.097+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:31.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:31.097+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:31.758 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:31.412+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:32.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:31.775+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:32.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:31.775+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:32.107 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:31.758+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:32.107 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:31 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:31.758+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:06.393821+0000 front 2024-04-14T15:30:06.393873+0000 (oldest deadline 2024-04-14T15:30:30.493668+0000) 2024-04-14T15:30:32.392 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:32.137+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:32.392 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:32.137+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:32.710 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:32.392+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:33.105 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:32.709+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.592735+0000 front 2024-04-14T15:30:00.592790+0000 (oldest deadline 2024-04-14T15:30:23.492683+0000) 2024-04-14T15:30:33.105 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4[38084]: 2024-04-14T15:30:32.709+0000 7f271c731700 -1 osd.4 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:06.393821+0000 front 2024-04-14T15:30:06.393873+0000 (oldest deadline 2024-04-14T15:30:30.493668+0000) 2024-04-14T15:30:33.105 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:32.791+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:33.105 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:32 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:32.791+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:33.357 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:33 smithi071 podman[69683]: 2024-04-14 15:30:33.349809232 +0000 UTC m=+5.846528937 container died 580a3e6d25fb7310c4c29d31f85cbdfca03696360b0d520aa793047aa7ef3b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-14T15:30:33.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:33.104+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:33.358 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:33.104+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:33.711 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:33.423+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:33.711 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:33.423+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:33.970 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:33 smithi071 podman[69683]: 2024-04-14 15:30:33.711916653 +0000 UTC m=+6.208636372 container cleanup 580a3e6d25fb7310c4c29d31f85cbdfca03696360b0d520aa793047aa7ef3b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-14T15:30:33.970 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:33 smithi071 bash[69683]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4 2024-04-14T15:30:33.971 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:33.781+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:33.971 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:33 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:33.781+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:33.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:33.574Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:33.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:33.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:33.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:33 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:33.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:34.274 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:33 smithi071 podman[69700]: 2024-04-14 15:30:33.971260499 +0000 UTC m=+0.621087820 container remove 580a3e6d25fb7310c4c29d31f85cbdfca03696360b0d520aa793047aa7ef3b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, org.label-schema.build-date=20240403) 2024-04-14T15:30:34.275 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:34.083+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:34.275 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:34.083+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:34.532 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:34.446+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:34.532 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:34.446+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:34.533 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.181876817 +0000 UTC m=+0.018413561 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:34.533 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.297363991 +0000 UTC m=+0.133900716 container create a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240403, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:34.533 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.533195603 +0000 UTC m=+0.369732328 container init a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-a971fd5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-14T15:30:34.836 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:34.740+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:34.836 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:34 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:34.740+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:34.837 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.536542063 +0000 UTC m=+0.373078791 container start a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:34.837 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.616359222 +0000 UTC m=+0.452895972 container attach a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-14T15:30:35.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:35.050+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:35.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:35.050+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:35.107 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:34 smithi071 podman[69794]: 2024-04-14 15:30:34.837220432 +0000 UTC m=+0.673757165 container died a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=main-a971fd5) 2024-04-14T15:30:35.403 INFO:journalctl@ceph.osd.4.smithi071.stdout:Apr 14 15:30:35 smithi071 podman[69819]: 2024-04-14 15:30:35.386339707 +0000 UTC m=+0.539651508 container remove a34309d3800ba99cd96b785fa22d2da3232af29e59465a639588b92eb75846f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-14T15:30:35.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.4.service' 2024-04-14T15:30:35.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:35.490 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-14T15:30:35.490 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-14T15:30:35.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.5 2024-04-14T15:30:35.676 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:35.413+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:35.676 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:35.413+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:35.676 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 systemd[1]: Stopping Ceph osd.5 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:36.051 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:35.709+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:36.051 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:35.710+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:36.051 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:35.839+0000 7f6ab4bca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:36.051 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:35.839+0000 7f6ab4bca700 -1 osd.5 773 *** Got signal Terminated *** 2024-04-14T15:30:36.051 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:35 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:35.839+0000 7f6ab4bca700 -1 osd.5 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:36.356 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:36.050+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:36.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:36.050+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:36.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:36.693+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:36.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:36.693+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:36.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:36.455+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:36.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:36 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:36.455+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:37.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:37.012+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:37.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:37.012+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:37.722 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:37.429+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:37.722 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:37.429+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:38.013 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:37.719+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:38.014 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:37 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:37.719+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:38.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:38.013+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:38.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:38.013+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:38.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:38.468+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:38.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:38.468+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:38.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:38.703+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:38.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:38.703+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:38.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:38 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:38.703+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:39.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:39.015+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:39.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:39.015+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:39.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:39.502+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:39.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:39.502+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:39.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:39.502+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:13.629053+0000 front 2024-04-14T15:30:13.629126+0000 (oldest deadline 2024-04-14T15:30:38.929012+0000) 2024-04-14T15:30:39.858 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:39.730+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:39.858 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:39.730+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:39.858 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:39.730+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:40.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:39.997+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:40.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:39.997+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:40.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:39 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:39.997+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:40.754 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:40.476+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:30:00.326923+0000 front 2024-04-14T15:30:00.326770+0000 (oldest deadline 2024-04-14T15:30:24.426683+0000) 2024-04-14T15:30:40.754 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:40.476+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:07.828201+0000 front 2024-04-14T15:30:07.828095+0000 (oldest deadline 2024-04-14T15:30:33.128096+0000) 2024-04-14T15:30:40.754 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5[43841]: 2024-04-14T15:30:40.476+0000 7f6ab11be700 -1 osd.5 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:13.629053+0000 front 2024-04-14T15:30:13.629126+0000 (oldest deadline 2024-04-14T15:30:38.929012+0000) 2024-04-14T15:30:41.019 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:40.753+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:41.019 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:40.753+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:41.019 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:40 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:40.753+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:41.289 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:41.019+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:41.289 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:41.019+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:41.290 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:41.019+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:41.557 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:41 smithi071 podman[69900]: 2024-04-14 15:30:41.289727009 +0000 UTC m=+5.569277218 container died f09325e1675b302ffda8f0c07d91d8685b31e40b60be49e5ae4c44a14f6ab26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240403, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-14T15:30:41.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:41 smithi071 podman[69900]: 2024-04-14 15:30:41.672671567 +0000 UTC m=+5.952221778 container cleanup f09325e1675b302ffda8f0c07d91d8685b31e40b60be49e5ae4c44a14f6ab26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, org.label-schema.build-date=20240403) 2024-04-14T15:30:41.857 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:41 smithi071 bash[69900]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5 2024-04-14T15:30:41.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:41.745+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:41.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:41.745+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:41.857 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:41 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:41.745+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:42.226 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:41 smithi071 podman[69929]: 2024-04-14 15:30:41.890337341 +0000 UTC m=+0.600767552 container remove f09325e1675b302ffda8f0c07d91d8685b31e40b60be49e5ae4c44a14f6ab26d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, ceph=True) 2024-04-14T15:30:42.226 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:42.035+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:42.226 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:42.035+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:42.226 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:42.035+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:42.484 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.13004647 +0000 UTC m=+0.018045268 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:42.484 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.257841006 +0000 UTC m=+0.145839804 container create dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:42.777 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.485171212 +0000 UTC m=+0.373170010 container init dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240403, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-14T15:30:42.777 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.488263912 +0000 UTC m=+0.376262715 container start dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5) 2024-04-14T15:30:42.777 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.568282507 +0000 UTC m=+0.456281328 container attach dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-14T15:30:43.046 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:42 smithi071 podman[70018]: 2024-04-14 15:30:42.777559048 +0000 UTC m=+0.665557847 container died dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-14T15:30:43.046 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:42.794+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:43.046 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:42.794+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:43.046 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:42 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:42.794+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:43.047 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.011+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:43.047 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.011+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:43.047 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.011+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:43.047 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.011+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:43.302 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 14 15:30:43 smithi071 podman[70044]: 2024-04-14 15:30:43.288504008 +0000 UTC m=+0.505533118 container remove dee589fa2f9d276499568553952a05f81342eea8502c47300a1da1c81248fc60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240403, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-14T15:30:43.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.5.service' 2024-04-14T15:30:43.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:43.347 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-14T15:30:43.348 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-14T15:30:43.348 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.6 2024-04-14T15:30:43.607 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 systemd[1]: Stopping Ceph osd.6 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:43.908 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:43.575Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:43.925 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:43.576Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:43.925 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:43 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:43.576Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:43.969 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.699+0000 7f84e03b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:43.970 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.699+0000 7f84e03b5700 -1 osd.6 773 *** Got signal Terminated *** 2024-04-14T15:30:43.970 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.699+0000 7f84e03b5700 -1 osd.6 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:43.970 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.787+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:43.970 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.787+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:43.970 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:43.787+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:44.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.969+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:44.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.969+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:44.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.969+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:44.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:43 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:43.969+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:44.778+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:44.778+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:44.778+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:44.989+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:44.989+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:44.989+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:45.107 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:44 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:44.989+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:46.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:45.823+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:46.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:45.823+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:46.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:45.823+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:46.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:45.979+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:46.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:45.979+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:46.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:45.980+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:46.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:45.980+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:46.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:45 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:45.980+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:47.002 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:46 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:46.800+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:47.003 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:46 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:46.800+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:47.003 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:46 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:46.800+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:47.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:47.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:47.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:47.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:47.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:47.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:47.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:47.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:47.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:47.003+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:48.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:47.845+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.885909+0000 front 2024-04-14T15:29:57.885813+0000 (oldest deadline 2024-04-14T15:30:21.385716+0000) 2024-04-14T15:30:48.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:47.845+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:05.788400+0000 front 2024-04-14T15:30:05.788438+0000 (oldest deadline 2024-04-14T15:30:29.288501+0000) 2024-04-14T15:30:48.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:47.845+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.190874+0000 front 2024-04-14T15:30:16.190843+0000 (oldest deadline 2024-04-14T15:30:37.890777+0000) 2024-04-14T15:30:48.107 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:47 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6[49517]: 2024-04-14T15:30:47.845+0000 7f84dc9a9700 -1 osd.6 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:21.292541+0000 front 2024-04-14T15:30:21.292470+0000 (oldest deadline 2024-04-14T15:30:47.192508+0000) 2024-04-14T15:30:48.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:48.033+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:48.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:48.033+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:48.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:48.033+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:48.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:48.033+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:48.108 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:48 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:48.034+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:49 smithi071 podman[70122]: 2024-04-14 15:30:49.215727381 +0000 UTC m=+5.643333152 container died a21dd893972673248c279dfb1e2fe5f61294ddda906c67b867cd9073d26fe32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, ceph=True, GIT_BRANCH=HEAD) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:49 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:49.054+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:49 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:49.054+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:49 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:49.054+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:49 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:49.054+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:49.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:49 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:49.054+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:49.757 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:49 smithi071 podman[70122]: 2024-04-14 15:30:49.565635882 +0000 UTC m=+5.993241642 container cleanup a21dd893972673248c279dfb1e2fe5f61294ddda906c67b867cd9073d26fe32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-14T15:30:49.758 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:49 smithi071 bash[70122]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6 2024-04-14T15:30:50.057 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:49 smithi071 podman[70140]: 2024-04-14 15:30:49.758398143 +0000 UTC m=+0.536492954 container remove a21dd893972673248c279dfb1e2fe5f61294ddda906c67b867cd9073d26fe32c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-a971fd5, maintainer=Guillaume Abrioux ) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.076172775 +0000 UTC m=+0.018059956 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.175737608 +0000 UTC m=+0.117624821 container create 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=main-a971fd5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:50 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:50.057+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:50 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:50.057+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:50 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:50.057+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:50 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:50.057+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:50.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:50 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:50.057+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:50.713 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.420308832 +0000 UTC m=+0.362196002 container init 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:50.713 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.423480687 +0000 UTC m=+0.365367859 container start 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-14T15:30:50.714 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.495009557 +0000 UTC m=+0.436896730 container attach 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-14T15:30:50.997 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:50 smithi071 podman[70230]: 2024-04-14 15:30:50.714534592 +0000 UTC m=+0.656421765 container died 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=) 2024-04-14T15:30:51.249 INFO:journalctl@ceph.osd.6.smithi071.stdout:Apr 14 15:30:51 smithi071 podman[70256]: 2024-04-14 15:30:51.248090876 +0000 UTC m=+0.522475461 container remove 0796a9d5695c9558a311ff6081873f151fa34ea5ba581f32222935303128c119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True) 2024-04-14T15:30:51.250 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.073+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:51.250 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.073+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:51.250 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.073+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:51.250 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.073+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:51.250 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.073+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:51.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.6.service' 2024-04-14T15:30:51.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:51.307 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-14T15:30:51.307 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-14T15:30:51.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.7 2024-04-14T15:30:51.607 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 systemd[1]: Stopping Ceph osd.7 for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:52.092 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.642+0000 7fd310b7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:52.092 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.642+0000 7fd310b7b700 -1 osd.7 773 *** Got signal Terminated *** 2024-04-14T15:30:52.092 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:51 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:51.642+0000 7fd310b7b700 -1 osd.7 773 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-14T15:30:52.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:52 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:52.091+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:52.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:52 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:52.091+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:52.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:52 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:52.091+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:52.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:52 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:52.091+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:52.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:52 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:52.092+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:53.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:53.103+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:53.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:53.103+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:53.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:53.103+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:53.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:53.103+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:53.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:53 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:53.103+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:53.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:53.576Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:53.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:53.577Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:30:53.997 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:30:53 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:30:53.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:30:54.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:54.063+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:54.358 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:54.063+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:54.358 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:54.063+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:54.358 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:54.063+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:54.358 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:54 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:54.063+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.71:6802 osd.4 since back 2024-04-14T15:30:28.690673+0000 front 2024-04-14T15:30:28.690651+0000 (oldest deadline 2024-04-14T15:30:54.590663+0000) 2024-04-14T15:30:55.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:55 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:55.020+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6803 osd.0 since back 2024-04-14T15:29:57.686800+0000 front 2024-04-14T15:29:57.686652+0000 (oldest deadline 2024-04-14T15:30:23.586603+0000) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6807 osd.1 since back 2024-04-14T15:30:03.587168+0000 front 2024-04-14T15:30:03.587191+0000 (oldest deadline 2024-04-14T15:30:29.487088+0000) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6811 osd.2 since back 2024-04-14T15:30:16.488529+0000 front 2024-04-14T15:30:16.488510+0000 (oldest deadline 2024-04-14T15:30:42.388462+0000) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.45:6815 osd.3 since back 2024-04-14T15:30:22.389056+0000 front 2024-04-14T15:30:22.389040+0000 (oldest deadline 2024-04-14T15:30:45.888908+0000) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 heartbeat_check: no reply from 172.21.15.71:6802 osd.4 since back 2024-04-14T15:30:28.690673+0000 front 2024-04-14T15:30:28.690651+0000 (oldest deadline 2024-04-14T15:30:54.590663+0000) 2024-04-14T15:30:56.357 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:56 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7[55575]: 2024-04-14T15:30:56.002+0000 7fd309167700 -1 osd.7 773 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24466.0:1195 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e773) 2024-04-14T15:30:57.332 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 podman[70335]: 2024-04-14 15:30:57.002632484 +0000 UTC m=+5.480081982 container died 5d18d19fbc825ee77f3ce58939cc02effcbb73d03f5656bd6f2279acadbb5a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-a971fd5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-14T15:30:57.607 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 podman[70335]: 2024-04-14 15:30:57.333242316 +0000 UTC m=+5.810691817 container cleanup 5d18d19fbc825ee77f3ce58939cc02effcbb73d03f5656bd6f2279acadbb5a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:57.607 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 bash[70335]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7 2024-04-14T15:30:57.607 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 podman[70364]: 2024-04-14 15:30:57.534243208 +0000 UTC m=+0.535425850 container remove 5d18d19fbc825ee77f3ce58939cc02effcbb73d03f5656bd6f2279acadbb5a15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-14T15:30:58.237 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 podman[70457]: 2024-04-14 15:30:57.935369605 +0000 UTC m=+0.104845660 container create 03e42e1e021a208ce0927fb2c53a10dbf7b1293d18468fde802bc0940da468f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-14T15:30:58.237 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:57 smithi071 podman[70457]: 2024-04-14 15:30:57.848115416 +0000 UTC m=+0.017591434 image pull b96c509eef841460b3271f5fdd331860da87b004524f4ed59414aa3680183b53 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608 2024-04-14T15:30:58.237 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:58 smithi071 podman[70457]: 2024-04-14 15:30:58.154343717 +0000 UTC m=+0.323819729 container init 03e42e1e021a208ce0927fb2c53a10dbf7b1293d18468fde802bc0940da468f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240403, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-a971fd5) 2024-04-14T15:30:58.237 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:58 smithi071 podman[70457]: 2024-04-14 15:30:58.157599691 +0000 UTC m=+0.327075702 container start 03e42e1e021a208ce0927fb2c53a10dbf7b1293d18468fde802bc0940da468f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=main-a971fd5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=) 2024-04-14T15:30:58.237 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:58 smithi071 podman[70457]: 2024-04-14 15:30:58.237310265 +0000 UTC m=+0.406786293 container attach 03e42e1e021a208ce0927fb2c53a10dbf7b1293d18468fde802bc0940da468f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:58.607 INFO:journalctl@ceph.osd.7.smithi071.stdout:Apr 14 15:30:58 smithi071 podman[70457]: 2024-04-14 15:30:58.450129147 +0000 UTC m=+0.619605157 container died 03e42e1e021a208ce0927fb2c53a10dbf7b1293d18468fde802bc0940da468f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-a971fd5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-14T15:30:59.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@osd.7.service' 2024-04-14T15:30:59.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:30:59.072 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-14T15:30:59.072 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-04-14T15:30:59.072 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@rgw.foo.a 2024-04-14T15:30:59.376 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:30:59 smithi045 systemd[1]: Stopping Ceph rgw.foo.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:30:59.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:30:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a[64137]: 2024-04-14T15:30:59.376+0000 7f8448934700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-14T15:30:59.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:30:59 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a[64137]: 2024-04-14T15:30:59.376+0000 7f84571bbac0 -1 shutting down 2024-04-14T15:31:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:30:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:30:59.753Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:31:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:30:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:30:59.755Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:31:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:30:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:30:59.758Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:31:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:30:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:30:59.758Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:31:00.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:30:59 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:30:59.758Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.45:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.45:8765: connect: connection refused" 2024-04-14T15:31:03.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:31:03.577Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:31:03.997 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:31:03.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi045.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.45:8443: connect: connection refused" 2024-04-14T15:31:03.997 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:03 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:31:03.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-04-14T15:31:09.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:31:09 smithi045 bash[147752]: time="2024-04-14T15:31:09Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-04-14T15:31:09.746 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:31:09 smithi045 podman[147752]: 2024-04-14 15:31:09.396081256 +0000 UTC m=+10.121375554 container died f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a, RELEASE=main-a971fd5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240403, ceph=True) 2024-04-14T15:31:10.092 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:31:09 smithi045 podman[147752]: 2024-04-14 15:31:09.850017829 +0000 UTC m=+10.575312124 container cleanup f64ee0003fd912c222e948e2b920b241f2fcdc57d27a381b3f029fcdb20b3016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-a971fd5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240403, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-14T15:31:10.092 INFO:journalctl@ceph.rgw.foo.a.smithi045.stdout:Apr 14 15:31:09 smithi045 bash[147752]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-rgw-foo-a 2024-04-14T15:31:10.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@rgw.foo.a.service' 2024-04-14T15:31:10.255 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:31:10.255 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-04-14T15:31:10.255 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-04-14T15:31:10.255 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@prometheus.a 2024-04-14T15:31:10.541 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 systemd[1]: Stopping Ceph prometheus.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-04-14T15:31:10.800 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-04-14T15:31:10.801 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-04-14T15:31:10.801 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-04-14T15:31:10.801 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a[65200]: ts=2024-04-14T15:31:10.541Z caller=main.go:1147 level=info msg="See you next time!" 2024-04-14T15:31:10.801 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 podman[70562]: 2024-04-14 15:31:10.55177858 +0000 UTC m=+0.108417941 container died 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:11.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 podman[70562]: 2024-04-14 15:31:10.907898598 +0000 UTC m=+0.464537966 container cleanup 51e5423cfaac7809f9414c4fd50875c892324fb1b9ba031515202bd3e688e343 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:11.107 INFO:journalctl@ceph.prometheus.a.smithi071.stdout:Apr 14 15:31:10 smithi071 bash[70562]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-prometheus-a 2024-04-14T15:31:11.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@prometheus.a.service' 2024-04-14T15:31:11.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-14T15:31:11.345 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-04-14T15:31:11.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm rm-cluster --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 --force --keep-logs 2024-04-14T15:31:11.542 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:31:13.246 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:13 smithi045 systemd[1]: Stopping Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:13.655 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:13 smithi045 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a[73740]: ts=2024-04-14T15:31:13.363Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-04-14T15:31:13.655 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:13 smithi045 podman[148242]: 2024-04-14 15:31:13.404010511 +0000 UTC m=+0.157107854 container died 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:13.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:13 smithi045 podman[148242]: 2024-04-14 15:31:13.75510054 +0000 UTC m=+0.508197932 container cleanup 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:13.996 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:13 smithi045 bash[148242]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a 2024-04-14T15:31:14.443 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:14 smithi045 podman[148256]: 2024-04-14 15:31:14.18924547 +0000 UTC m=+0.788828290 container remove 30ce3652b2d963e43ed4fb7b95cfbd1b503bc5c55a9e2a2712afb018b0ea204a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:14.444 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:14 smithi045 podman[148256]: 2024-04-14 15:31:14.265228527 +0000 UTC m=+0.864811353 volume remove 8a1a04bc4e9e519294b6465f812ca4b1029ba482b87c8b113aadd3ae120ba8d2 2024-04-14T15:31:14.444 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:14 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-04-14T15:31:14.444 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:14 smithi045 systemd[1]: Stopped Ceph alertmanager.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:31:14.444 INFO:journalctl@ceph.alertmanager.a.smithi045.stdout:Apr 14 15:31:14 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@alertmanager.a.service: Consumed 1.744s CPU time. 2024-04-14T15:31:14.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:14 smithi045 systemd[1]: Stopping Ceph node-exporter.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:15.134 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:14 smithi045 podman[148369]: 2024-04-14 15:31:14.87781429 +0000 UTC m=+0.117500968 container died f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:15.476 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 podman[148369]: 2024-04-14 15:31:15.234036687 +0000 UTC m=+0.473723347 container cleanup f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:15.476 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 bash[148369]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a 2024-04-14T15:31:15.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 podman[148383]: 2024-04-14 15:31:15.476736037 +0000 UTC m=+0.602462997 container remove f38a015bbdf02a63415a9f43284af32e0917b28de20e99844d89420c1f0f7149 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-04-14T15:31:15.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-04-14T15:31:15.746 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-04-14T15:31:15.747 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 systemd[1]: Stopped Ceph node-exporter.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:31:15.747 INFO:journalctl@ceph.node-exporter.a.smithi045.stdout:Apr 14 15:31:15 smithi045 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.a.service: Consumed 3.606s CPU time. 2024-04-14T15:31:16.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm rm-cluster --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 --force --keep-logs 2024-04-14T15:31:16.482 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:31:17.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:17 smithi071 systemd[1]: Stopping Ceph iscsi.iscsi.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:18.358 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:17 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a[60191]: debug Shutdown received 2024-04-14T15:31:28.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:27 smithi071 bash[70976]: time="2024-04-14T15:31:27Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-04-14T15:31:28.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:27 smithi071 podman[70976]: 2024-04-14 15:31:27.981633759 +0000 UTC m=+10.120676408 container died dce9ae0da5cf1b4891a9e790d786aaf05f41dbe434872aca061e90eec4e3129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240403, org.label-schema.schema-version=1.0, RELEASE=main-a971fd5, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-14T15:31:28.601 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 podman[70976]: 2024-04-14 15:31:28.342791474 +0000 UTC m=+10.481834112 container cleanup dce9ae0da5cf1b4891a9e790d786aaf05f41dbe434872aca061e90eec4e3129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, RELEASE=main-a971fd5, org.label-schema.build-date=20240403, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-14T15:31:28.601 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 bash[70976]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a 2024-04-14T15:31:28.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 podman[70991]: 2024-04-14 15:31:28.602224597 +0000 UTC m=+0.618013795 container remove dce9ae0da5cf1b4891a9e790d786aaf05f41dbe434872aca061e90eec4e3129c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:02f0a191d4d693952654c64f93cc9c0152602c0915251b3c94c2c4e32855c608, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240403, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-a971fd5) 2024-04-14T15:31:28.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-04-14T15:31:28.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-04-14T15:31:28.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 systemd[1]: Stopped Ceph iscsi.iscsi.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:31:28.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi071.stdout:Apr 14 15:31:28 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@iscsi.iscsi.a.service: Consumed 3.363s CPU time. 2024-04-14T15:31:30.424 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 systemd[1]: Stopping Ceph grafana.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:30.684 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=server t=2024-04-14T15:31:30.426560955Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-04-14T15:31:30.684 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=tracing t=2024-04-14T15:31:30.426977589Z level=info msg="Closing tracing" 2024-04-14T15:31:30.684 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a[63848]: logger=ticker t=2024-04-14T15:31:30.427010299Z level=info msg=stopped last_tick=2024-04-14T15:31:30Z 2024-04-14T15:31:30.684 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 podman[71258]: 2024-04-14 15:31:30.436862706 +0000 UTC m=+0.106588036 container died 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:31:30.942 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 podman[71258]: 2024-04-14 15:31:30.75864176 +0000 UTC m=+0.428367090 container cleanup 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:31:30.942 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 bash[71258]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a 2024-04-14T15:31:31.201 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:30 smithi071 podman[71272]: 2024-04-14 15:31:30.943090297 +0000 UTC m=+0.503536626 container remove 1f99e6c0d0dae4366211c3fd2d8a46d31248f0cdfd95a7a8a9800311c9009089 (image=quay.io/ceph/grafana:9.4.12, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-04-14T15:31:31.201 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:31 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@grafana.a.service: Deactivated successfully. 2024-04-14T15:31:31.202 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:31 smithi071 systemd[1]: Stopped Ceph grafana.a for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:31:31.202 INFO:journalctl@ceph.grafana.a.smithi071.stdout:Apr 14 15:31:31 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@grafana.a.service: Consumed 14.520s CPU time. 2024-04-14T15:31:31.607 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:31 smithi071 systemd[1]: Stopping Ceph node-exporter.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968... 2024-04-14T15:31:31.907 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:31 smithi071 podman[71394]: 2024-04-14 15:31:31.652479465 +0000 UTC m=+0.117981488 container died 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:31:32.265 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 podman[71394]: 2024-04-14 15:31:32.006120258 +0000 UTC m=+0.471622268 container cleanup 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:31:32.265 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 bash[71394]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b 2024-04-14T15:31:32.570 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 podman[71408]: 2024-04-14 15:31:32.265216617 +0000 UTC m=+0.613710248 container remove 7b64272bb80a9e4c2e79b5daae26f5a098762f16279b3de05e84b58cb135ce1e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-04-14T15:31:32.571 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-04-14T15:31:32.571 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-04-14T15:31:32.571 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 systemd[1]: Stopped Ceph node-exporter.b for 1eba33fc-fa70-11ee-bc8e-c7b262605968. 2024-04-14T15:31:32.571 INFO:journalctl@ceph.node-exporter.b.smithi071.stdout:Apr 14 15:31:32 smithi071 systemd[1]: ceph-1eba33fc-fa70-11ee-bc8e-c7b262605968@node-exporter.b.service: Consumed 3.446s CPU time. 2024-04-14T15:31:33.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:31:33.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-14T15:31:33.113 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-14T15:31:33.115 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi045/crash 2024-04-14T15:31:33.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash -- . 2024-04-14T15:31:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash: Cannot open: No such file or directory 2024-04-14T15:31:33.163 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-04-14T15:31:33.165 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi071/crash 2024-04-14T15:31:33.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash -- . 2024-04-14T15:31:33.206 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/crash: Cannot open: No such file or directory 2024-04-14T15:31:33.206 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-04-14T15:31:33.207 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-14T15:31:33.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-14T15:31:33.241 INFO:teuthology.orchestra.run.smithi045.stdout:2024-04-14T15:20:00.000124+0000 mon.a (mon.0) 2648 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-04-14T15:31:33.241 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-14T15:31:33.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[SEC\]' /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-14T15:31:33.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]' /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-14T15:31:33.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[WRN\]' /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-04-14T15:31:33.458 INFO:teuthology.orchestra.run.smithi045.stdout:2024-04-14T15:20:00.000124+0000 mon.a (mon.0) 2648 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-04-14T15:31:33.458 INFO:tasks.cephadm:Compressing logs... 2024-04-14T15:31:33.459 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-14T15:31:33.501 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.a.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-client.rgw.foo.a.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.1.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.3.log 2024-04-14T15:31:33.527 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.y.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.6.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.4.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.7.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.5.log 2024-04-14T15:31:33.531 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.x.log 2024-04-14T15:31:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/tcmu-runner.log 2024-04-14T15:31:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log 2024-04-14T15:31:33.532 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/tcmu-runner.log: 67.2% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/tcmu-runner.log.gz 2024-04-14T15:31:33.535 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log 2024-04-14T15:31:33.535 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log.gz 2024-04-14T15:31:33.535 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log 2024-04-14T15:31:33.536 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.1.log: /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.0.log 2024-04-14T15:31:33.536 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.y.log: /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log 2024-04-14T15:31:33.537 INFO:teuthology.orchestra.run.smithi045.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-14T15:31:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log 2024-04-14T15:31:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log: 89.1% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.cephadm.log.gz 2024-04-14T15:31:33.537 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.2.log 2024-04-14T15:31:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log: 87.1% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log.gz 2024-04-14T15:31:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.b.log 2024-04-14T15:31:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log: 92.4% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log.gz 2024-04-14T15:31:33.540 INFO:teuthology.orchestra.run.smithi071.stderr: 91.7% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.x.log.gz/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.b.log: 2024-04-14T15:31:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log: 92.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.log.gz 2024-04-14T15:31:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log 2024-04-14T15:31:33.542 INFO:teuthology.orchestra.run.smithi071.stderr: 91.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-14T15:31:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.2.log: 95.3% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph.audit.log.gz 2024-04-14T15:31:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.c.log 2024-04-14T15:31:33.552 INFO:teuthology.orchestra.run.smithi071.stderr: 90.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log.gz 2024-04-14T15:31:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log: /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-volume.log.gz 2024-04-14T15:31:33.569 INFO:teuthology.orchestra.run.smithi045.stderr: 93.0% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-04-14T15:31:33.950 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mgr.y.log.gz 2024-04-14T15:31:34.695 INFO:teuthology.orchestra.run.smithi071.stderr: 92.6% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.b.log.gz 2024-04-14T15:31:34.915 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.c.log.gz 2024-04-14T15:31:36.511 INFO:teuthology.orchestra.run.smithi045.stderr: 91.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-mon.a.log.gz 2024-04-14T15:31:39.943 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.2.log.gz 2024-04-14T15:31:39.966 INFO:teuthology.orchestra.run.smithi071.stderr: 93.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.6.log.gz 2024-04-14T15:31:40.102 INFO:teuthology.orchestra.run.smithi071.stderr: 93.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.5.log.gz 2024-04-14T15:31:40.362 INFO:teuthology.orchestra.run.smithi071.stderr: 93.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.7.log.gz 2024-04-14T15:31:40.412 INFO:teuthology.orchestra.run.smithi045.stderr: 93.8% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.0.log.gz 2024-04-14T15:31:40.697 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.3.log.gz 2024-04-14T15:31:40.715 INFO:teuthology.orchestra.run.smithi071.stderr: 93.9% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.4.log.gz 2024-04-14T15:31:40.717 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-14T15:31:40.717 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m7.201s 2024-04-14T15:31:40.717 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m27.271s 2024-04-14T15:31:40.717 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.947s 2024-04-14T15:31:41.350 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/1eba33fc-fa70-11ee-bc8e-c7b262605968/ceph-osd.1.log.gz 2024-04-14T15:31:41.352 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-14T15:31:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m7.839s 2024-04-14T15:31:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m32.067s 2024-04-14T15:31:41.352 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m1.033s 2024-04-14T15:31:41.352 INFO:tasks.cephadm:Archiving logs... 2024-04-14T15:31:41.353 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi045/log 2024-04-14T15:31:41.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-14T15:31:45.626 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi071/log 2024-04-14T15:31:45.628 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-14T15:31:49.538 INFO:tasks.cephadm:Removing cluster... 2024-04-14T15:31:49.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo cephadm rm-cluster --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 --force 2024-04-14T15:31:49.740 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:31:50.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo cephadm rm-cluster --fsid 1eba33fc-fa70-11ee-bc8e-c7b262605968 --force 2024-04-14T15:31:50.212 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 1eba33fc-fa70-11ee-bc8e-c7b262605968 2024-04-14T15:31:50.503 INFO:tasks.cephadm:Teardown complete 2024-04-14T15:31:50.503 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-14T15:31:50.513 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-14T15:31:50.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-14T15:31:50.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-14T15:31:50.595 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-14T15:31:50.603 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-14T15:31:50.603 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T15:31:50.606 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-14T15:31:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-14T15:31:50.620 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-14T15:31:50.622 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T15:31:50.622 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-14T15:31:50.622 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +987us[ +988us] +/- 54ms 2024-04-14T15:31:50.622 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -1739us[-1738us] +/- 75ms 2024-04-14T15:31:50.623 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +388us[ +388us] +/- 76ms 2024-04-14T15:31:50.623 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +980us[ +978us] +/- 54ms 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -1761us[-1761us] +/- 75ms 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +383us[ +381us] +/- 76ms 2024-04-14T15:31:50.624 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-14T15:31:50.625 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-14T15:31:50.637 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-14T15:31:50.638 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-14T15:31:50.672 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-14T15:31:50.749 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-14T15:31:50.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-14T15:31:50.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T15:31:50.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-04-14T15:31:50.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T15:31:50.886 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-14T15:31:50.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-14T15:31:50.922 INFO:teuthology.task.internal:Duration was 2183.025375 seconds 2024-04-14T15:31:50.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-14T15:31:50.955 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-14T15:31:50.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-14T15:31:50.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-14T15:31:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T15:31:51.001 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-14T15:31:51.306 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-14T15:31:51.306 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-14T15:31:51.307 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-14T15:31:51.366 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-14T15:31:51.366 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-14T15:31:51.394 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-14T15:31:51.395 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-14T15:31:51.409 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-14T15:31:51.463 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-14T15:31:51.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-14T15:31:51.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-14T15:31:51.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-14T15:31:51.899 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-14T15:31:51.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-14T15:31:51.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-14T15:31:51.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-14T15:31:51.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-14T15:31:51.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-14T15:31:51.992 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-14T15:31:52.019 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-14T15:31:52.058 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-14T15:31:52.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T15:31:52.073 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-14T15:31:52.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-14T15:31:52.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-14T15:31:52.123 INFO:teuthology.task.internal:Transferring archived files... 2024-04-14T15:31:52.124 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi045 2024-04-14T15:31:52.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-14T15:31:52.189 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060/remote/smithi071 2024-04-14T15:31:52.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-14T15:31:52.228 INFO:teuthology.task.internal:Removing archive directory... 2024-04-14T15:31:52.228 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-14T15:31:52.231 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-14T15:31:52.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-14T15:31:52.290 INFO:teuthology.task.internal:Not uploading archives. 2024-04-14T15:31:52.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-14T15:31:52.314 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-14T15:31:52.314 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-14T15:31:52.317 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-14T15:31:52.328 INFO:teuthology.orchestra.run.smithi045.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 14 15:31 /home/ubuntu/cephtest 2024-04-14T15:31:52.331 INFO:teuthology.orchestra.run.smithi071.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 14 15:31 /home/ubuntu/cephtest 2024-04-14T15:31:52.333 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-14T15:31:52.348 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-14T15:31:52.423 INFO:teuthology.nuke:Checking targets against current locks 2024-04-14T15:31:52.449 DEBUG:teuthology.nuke:shortname: smithi045 2024-04-14T15:31:52.450 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T15:31:52.472 DEBUG:teuthology.nuke:shortname: smithi071 2024-04-14T15:31:52.472 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-14T15:31:52.476 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 14:47:02.453171', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIgw8XwaCi5zWKb5A5BMOx1xFLxK4BHifsyKLM7h5Co0jY5I4GGsxgFWR4Vj/Yj3l12z506Fc5Z38/URcB+vWw='} 2024-04-14T15:31:52.500 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-08_20:08:14-orch-main-distro-default-smithi/7646060', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-14 14:47:02.451179', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPRRi40gafZuRBeII8+chWhAdPK7DY92ZmEkVZmZuS+dKcKDeSShFwO2r8zcYdbgZVmIRbxW1lUF6djnR096H/s='} 2024-04-14T15:31:52.527 INFO:teuthology.orchestra.console.smithi045:Power off 2024-04-14T15:31:52.527 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T15:31:52.553 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-04-14T15:31:52.553 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:31:52.568 INFO:teuthology.orchestra.console.smithi071:Power off 2024-04-14T15:31:52.568 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T15:31:52.579 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-14T15:31:52.590 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-04-14T15:31:52.590 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:31:52.615 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-14T15:31:56.581 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:31:56.616 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:31:56.692 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-14T15:31:56.725 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-04-14T15:32:00.697 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:32:00.727 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T15:32:00.808 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-14T15:32:00.840 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-04-14T15:32:00.909 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-04-14T15:32:00.941 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-04-14T15:32:01.045 INFO:teuthology.run:Summary data: description: orch/cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 2183.025375366211 failure_reason: '"2024-04-14T15:20:00.000124+0000 mon.a (mon.0) 2648 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-04-14T15:32:01.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T15:32:01.135 INFO:teuthology.run:FAIL